A General Framework for Mixed and Incomplete Data Clustering Based on Swarm Intelligence Algorithms

Abstract

Swarm intelligence has appeared as an active field for solving numerous machine-learning tasks. In this paper, we address the problem of clustering data with missing values, where the patterns are described by mixed (or hybrid) features. We introduce a generic modification to three swarm intelligence algorithms (Artificial Bee Colony, Firefly Algorithm, and Novel Bat Algorithm). We experimentally obtain the adequate values of the parameters for these three modified algorithms, with the purpose of applying them in the clustering task. We also provide an unbiased comparison among several metaheuristics based clustering algorithms, concluding that the clusters obtained by our proposals are highly representative of the “natural structure” of data.

1. Introduction

Clustering has become crucial in several areas of scientific research, and it has also a significant impact in the theoretical development and applied research in several scientific disciplines [1,2,3]. Despite the very large number of methods to perform clustering, the use of swarm intelligence algorithms has become increasingly relevant in order to perform this task [3,4,5,6]. However, given the ample diversity of fields, topics, and problems studied, a particular phenomenon may very well be described by both numerical and categorical variables (mixed or hybrid data). Additionally, the appearance of missing values has become increasingly common in data measurement and sampling processes (missing data). There are several actors that can cause the missing values. Among them, the most important ones can be mentioned: the impossibility to perform some measurements, the loss of already taken samples, or even the non-existence of information about the data being described.

Data described by hybrid or mixed features (or simply mixed data) represent a challenge to most automatic learning algorithms. This is due to the generally accepted assumption of data are described by features of the same kind. Consequently, most pattern recognition algorithms are designed to tackle problems whose attributes are all the same kind. Thus, some methods assume the existence of a metric space (e.g., k-Means algorithm [7]), while other clustering algorithms require the instances to be described only by categorical features (e.g., Partitioning Around Medoids PAM [8]).

Unlike mixed data, missing or incomplete data do not depend on the features describing the phenomenon of interest, but rather appear due to the lack of the value for a particular attribute, on a specific instance. A particular value of a specific object may be unknown for numerous causes. Among the most frequent are: the inability to measure the desired variable; lack of knowledge regarding the phenomenon of interest [9] (commonly occurring in social sciences, where the data represents people under study); and the loss of a previously acquired value [10] (e.g., due to data sampling or storing equipment failure, contamination, or data loss during transmission).

It is commonly agreed upon by the scientific community of pattern recognition that the presence of missing values in a particular dataset implies an additional challenge for automatic learning algorithms. This is due to the prevailing trend of designing such methods to handle whole datasets: that is, with no missing information.

On the other hand, most of the proposed algorithms for clustering operate only over numerical data [2,3,5,6,11,12,13,14]. This unfortunate situation causes that several applications, which are relevant for human activities and whose data are described not only by numeric attributes, cannot be solved effectively.

Considering the previously described scenario, the methods and algorithm for clustering over mixed and incomplete data, are an evident minority in scientific literature [15]. The above considerations show the need of carried out thoughtful and systematic scientific research for the creation, design, implementation and application of methods and algorithms to perform intelligent clustering over mixed and incomplete data. The proposal of this paper clearly aligns along with the presented need.

In this paper, we use swarm intelligence algorithms, which are applied to clustering mixed and incomplete data in a unified and satisfactory approach. In addition, we perform a large amount of thoughtful and systematics numerical experiments. It is well-known among specialized swarm intelligence researchers that those algorithms are overly sensitive to parameter configuration; however, despite this being a highly relevant issue, to date we cannot find systematic studies analyzing the sensibility to parameter configuration for clustering trough swarm intelligence algorithms. We also address this issue in this paper.

The obtained results allow us to determine experimentally the adequate parameter configuration to obtain clusters of high quality over mixed and incomplete data. The above is done in such a way that the comparative studies performed allow us to state that, beyond doubt, the results obtained in this research are superiors to others reported in the literature.

The rest of the paper is organized as follows: Section 2 reviews some previous works for mixed and incomplete data clustering. Section 3 explains the proposed generic framework for swarm intelligence algorithms for clustering of mixed and incomplete data. Section 4 explores some case of study, and Section 5 addresses the experimental outline, the obtained results, and discusses them. The article ends with the conclusions and future works.

2. Background

Handling patterns described by mixed data or those who include missing values represents an additional challenge to tackle for automatic learning algorithms. The problem of mixed data (also known as hybrid data) can be defined as follows:

Let

$\mathrm{X}=\left\{x_1,x_2,\cdots ,x_{ds}\right\}$ be a dataset (i.e., set of objects, instances, or samples) in a universe U, where each object is described by a set of features or attributes $\mathrm{A}=\left\{A_1,A_2,\cdots ,A_m\right\}$; each feature $A_p$ has associated a definition domain $dom\left(A_p\right)$, which in turn may be of Boolean, k-valued, integer, real, or another kind [16].

Usually, the lack of information is denoted with the symbol “?” in many datasets available in public repositories, such as the UCI Machine Learning Repository [17] or the KEEL Project Repository [18,19]. Following this criterion, the value “?” is included in the domain of definition of the dataset feature; thus, the description of an incomplete object has the value “?” in place of the missing value. On the other hand, it is entirely possible that hybrid features, while also presenting missing values for some patterns describe a dataset. In this instance, the challenge faced by classification algorithms is even greater, since they need to handle both problems.

The solutions developed to manage such situations can be grouped mainly in two general strategies: they may work either by modifying the dataset before presenting it to the classification method; or by giving a particular algorithm the mechanisms to work with hybrid data, missing values, or both. The current section is dedicated to discussing some of the most well-known alternatives for solutions to one or both problems.

2.1. Data Level Solution for Clustering Mixed and Incomplete Data

Many automatic learning algorithms are designed to work only with numerical data, or only with categorical data, or only with complete data (i.e., no missing values). Thus, data-level solutions to these problems by means of pre-processing techniques [20] are primarily focused on one (or several, depending on the situation) of the following tasks: coding categorical data into a numerical representation, discretizing numerical data into a categorical representation, or solving missing values by eliminating or completing the corresponding patterns.

Despite the numerous advances in this regard, every data-level solution has an unavoidable impact on the dataset, since they will inevitably transform the data and even alter the intrinsic semantic relationships associated to the original representation of such data. This, in turn, has given rise to a spreading opinion among researchers and practitioners, in the sense that a more adequate solution to the problems of mixed data and missing values rests in developing automatic learning algorithms able to internally handle both aspects.

2.2. Algorithm Level Solution for Clustering Mixed and Incomplete Data

In this approach, the responsibility of handling mixed and incomplete data representations falls on the learning algorithm, considering that said method must include mechanisms aimed at such kinds of data. A recent review of clustering algorithms for mixed data can be found in [15].

Most of the algorithms for clustering mixed data rely on dissimilarity functions able to deal with such kind of data [21]. This approach backs to 1997, when Huang proposed the k-Prototype algorithm [22]. k-Prototype is an extension of the k-Means algorithm [23]. The extension is based on the definition of a dissimilarity function to compare the mixed instances, and in a new strategy to compute the cluster centroids.

All such mixed dissimilarity-based algorithms, as k-Prototype, do not consider the attribute dependences. This is due to these algorithms analyze the numerical and categorical features separately, and some of them separately construct the clusters centroids. Finally, the use of arbitrarily dissimilarity function may be inadequate for some domains.

Another strategy for mixed data clustering is to separately analyze the features, by projecting the instances. That is, the mixed dataset is divided into several subsets according to the different types of attributes, and each subset is clustered using according to some algorithms, and then, the results are combined as a new categorical dataset and then are clustered [24]. This idea has an obvious impact on the dataset, since they will transform the data twice and even alter the intrinsic semantic relationships associated with the original representation of such data.

A combination of the above alternatives is also presented in the literature. Some algorithms do divide the data, and also use mixed dissimilarity functions. By doing this, the disadvantages of both strategies are conserved. An example is the HyDaP algorithm [25]. It has two steps. The first one involves identifying the data structure formed by continuous variables and recognizing the relevant features for clustering. Then, the second step involves using a dissimilarity measure for mixed data to obtain clustering results.

In our opinion, a better solution can be found by applying bio-inspired algorithms. To the best of our knowledge, the AGKA algorithm by Roy and Sharma [4] is the first algorithm using bio-inspired strategies for dealing with the clustering of mixed data. It uses a genetic algorithm to obtain the clusters.

The representation used is a string having as length the instance count, where the i-th element of the string contains the cluster index to which the i-th instance is assigned. To obtain the cluster centers, the AGKA algorithm uses the centroid representation strategy and the dissimilarity function proposed in [21].

Despite the advantage of considering an evolutionary approach to clustering, the use of a pre-defined dissimilarity function is a drawback of the AGKA algorithm. In addition, the centroid description used may be inadequate to obtain few clusters, due to the characteristics of the cluster centroid description. On the other hand, the representation used by the AGKA algorithm makes difficult for the application of traditional operators to evolve the individuals. In addition, having thousands of instances implies having individuals described by string of thousands of lengths, which makes AGKA inadequate to handle medium and big size data.

To solve these drawbacks, we propose a generic framework for dealing with mixed and incomplete data clustering, in a more suitable way.

3. Generic Framework for Bio-Inspired Clustering of Mixed and Incomplete Data

Swarm intelligence is directly related to the intelligent behavior of collections of individuals or agents (birds, ants, fish, bats, bees, etc.) that move in an apparently unorganized way. In this context, this branch of scientific research aims to develop processes and algorithms that model and simulate the actions that these individuals perform to search for food [3,4].

There is an impressive range of possibilities for scientists to choose the types of agents that exhibit cooperative and self-organizing behavior. In addition to insects, birds and fish, it is also possible to consider the growth of bacteria or even swarms of robots that make up cellular robotic systems [5].

Typically, the individuals in the swarm are simple entities, but active and with their own movements. These individual movements, when incorporated into the swarm, allow generating a cooperative movement that results in collective search strategies, which significantly improve random searches. This is precisely called swarm intelligence, whose rules are so simple that it is a cause for wonder to realize how efficiently searches are performed [6,11].

In the following, we will supply a generic framework for mixed data clustering using bio-inspired algorithms. We propose a unified representation of the solutions, and a strategy to update the clustering solutions in the optimization process. Our approach is quite generic, and we believe that our proposal can be easily applied to several bio-inspired algorithms. It should be noted that we only focus on iterative improvement metaheuristics, not constructive ones. Constructive algorithms obtain the desired solution by parts. That is, there is no solution to the problem until the algorithm finishes. On the other hand, improvement heuristics start with an entire solution or set of solutions (usually random), and then refine them in the iterative process. This characteristic allows the user to have at least one solution to the problem at every iteration of the algorithm.

Our rationale is that using a unified strategy to model the clustering problem as an optimization problem, will lead to good clustering results, despite the swarm intelligence algorithm used. We bet on selecting a suitable representation, a useful updating strategy and an adequate optimization function. Our hypothesis is that if we manage to model mixed and incomplete data clustering as an optimization problem, we can obtain competitive results, and we will be able to obtain a clustering that fits the natural structure of data.

3.1. Representation

The representation of candidate solutions is one of the key aspects in metaheuristic algorithms. In fact, the representation used defines the operators applicable to the candidate solutions. For example, in real-valued representations, it is possible to consider a new solution by increasing the value of the current one with an epsilon value. In binary representation, the only modification allowed is the bit changing. In order-related representations, a solution is changed or updated by swapping some of its elements (ex. 2-opt operators [26]).

For clustering numerical data, the usual representation has been a matrix of size $k\times m$, where $k$ is the number of clusters and $m$ is the number of feature values [27]. However, as we are dealing with mixed values, with some absences on information, we adopted a simpler approach: we consider a candidate solution as an array of cluster centers.

Instead of creating an artificial center of a cluster $C_l$, with $l\in \left[1,k\right]$, which is a clear disadvantage of previous proposals, we used the notion of prototype, in which the cluster center $\overline{c_i}$ is defined as the instance who minimizes the dissimilarity with respect to the rest of instances in the cluster. It is not worthy to mention that we can use any dissimilarity function in the algorithms. By doing this, we also solve the drawback of using predefined functions. In Figure 1, it is shown an example of computing the center of a cluster, using the Euclidean distance.

Formally, let $diss\left(p,q\right)$ be the dissimilarity among instances $p$ and $q$, belonging to a cluster $C_l$. The cluster center $\overline{c_i}$ is selected as following:

$$\overline{c_i}=\arg \underset{}{\min }\left\{{{\displaystyle \sum }}_{\begin{array}{c}p,qϵC_l\\ p\ne q\end{array}}diss\left(p,q\right)\right\}$$

(1)

If the minimum dissimilarity value is obtained with more than one instance, we can select as cluster center any of the instances that minimize the dissimilarity.

Thus, the proposed representation is an array, formed by the selected center $\overline{c_i}$ of each cluster $C_l$. Each individual $S_i$ is represented as $S_i=\left[\overline{c_{i1}}, \overline{c_{i2}}, \dots , \overline{c_{ik}}\right]$. Considering this representation, an individual will have its current location (array of cluster centers) as well as other algorithm-dependent parameters. The swarm will know the set of instances to be clustered, and the optimization functions associated to the clustering problem.

From an implementation perspective, we can easily simplify this representation, by using the indexes of the selected cluster center instead of the information of the cluster center itself. Figure 2 shows the implementation of the proposed representation of a candidate solution, using the indices instead of the feature values of the cluster centers.

This representation is simple, suitable, and easy to update, fulfilling the desirable characteristics for candidate solution representation in optimization problems. We consider that this proposal simplifies the optimization process, and is computationally effective, due to it consumes lower storage memory (just $k$) than the usual $k\times m$ matrix representation. It is important to note than some problems have a large number of instances (i.e., m parameter), on the order of thousands and even hundreds of thousands, which makes the traditional $k\times m$ representation computationally impractical. Our proposed representation contributes to computational efficiency, by diminishing the storage cost of the solution from $k\times m$ to just $k$.

3.2. Updating Strategy

All iterative improvement metaheuristic algorithms have in common that they update the current solution (or solutions) during the iterative cycle. The updating of the solutions usually takes into consideration the current solution, another solution in the population, the best solution in the population, and perhaps some randomly generated number (most programming languages include the capability of generating random number, usually by following a Uniform distribution. Unless specified otherwise, all random numbers referred in this paper follow a uniform distribution.). In this paper, we propose a unified updating strategy for dealing with the mixed clustering problem. Our proposal consists of changing a cluster center in the current solution. We generate a random number, representing the cluster whose center will be changed. Then, we selected from this cluster a random instance to replace the cluster center (Figure 3).

This strategy, although very simple, has two main advantages. It maintains a balance between exploration (by considering each instance in a cluster to be the new center) and exploitation (by considering only the instances already in the cluster to be center); and it directly handles the presence of attribute dependence, by selecting existent instances instead of independently modifying the features values of the centers.

Considering this updating strategy, it is possible to integrate it in the iterative improvement metaheuristic algorithms, while preserving the major characteristics of each of the original algorithms. In addition, this strategy is computationally simple due to it consisting only of updating the cluster centers.

In our implementation, we only need to generate a random number in the $\left[1,k\right]$ interval, which is very fast. In addition, because we only have a solution of size $k$ instead of a $k\times m$ matrix, our procedure is inexpensive. The pseudocode of the proposed updating procedure is detailed in Algorithm 1. This simple procedure will be used in the three analyzed swarm intelligence algorithms.

Algorithm 1. Pseudo code of the Updating procedure in the proposed framework to clustering mixed data.

Updating procedure

Inputs: $current$: individual; $X$: dataset of instances

Output: $new$: modified individual

Steps: $new=current$ Assign each instance of $X$ to its nearest cluster, considering the centers of new Generate a random number in the interval $\left[1, k\right]$ as $rdm\left(1,k\right)$. This number represent the index of the cluster center to be changed, and is defined as $idx\_c$. Let be $Instance{s}_i$ the array of instances of $X$ assigned to the i-th cluster. Generate a random number in the interval $\left[1,lenght\left(Instance{s}_{idx\_c}\right)\right]$ as $rdm(1,lenght\left(Instance{s}_{idx\_c}\right)$. This number represent the index of the instance replacing the cluster center, and is defined as $idx\_i$ $new\left[idx\_c\right]=Instance{s}_{idx\_C}\left[idx\_i\right]$ Return $new$

3.3. Fitness Functions and Dissimilarities

It has been assumed so far that there is a dissimilarity measure to compute the dissimilarity between instances in the clustering process. For experimental purposes, in this research, we used the Heterogeneous Euclidean Overlap Metric HEOM dissimilarity [28] to compare instances in such way. The HEOM dissimilarity was introduced by Wilson and Martínez [28] as an attempt to compare objects having mixed numerical and categorical descriptions. Let there be two instances $p$ and $q$, and the HEOM dissimilarity is defined as:

$$\begin{gathered} HEOM\left(p,q\right)=\sqrt{{\displaystyle \sum }_{A_i\in A}{d}_i{\left(p_i,q_i\right)}^2} \\ {d}_i\left(p_i,q_i\right)=\left\{\begin{array}{cc}1& \mathrm{if} p_i\vee q_i \mathrm{are} \mathrm{missing}\\ overlap\left(p_i,q_i\right)& \mathrm{if} A_i \mathrm{is} \mathrm{categoric}\\ rn\_diff\left(p_i,q_i\right)& \mathrm{if} A_i \mathrm{is} \mathrm{numeric}\end{array}\right. \\ overlap\left(p_i,q_i\right)=\left\{\begin{array}{cc}0& \mathrm{if} p_i=q_i \\ 1& \mathrm{otherwise}\end{array}\right., rn\_diff=\frac{\left|p_i-q_i\right|}{ma{x}_i- mi{n}_i} \end{gathered}$$

(2)

By using the HEOM dissimilarity, we are able to directly compare mixed and incomplete instances, and to carried out the clustering process and to evaluate the resulting clustering, with the selected optimization functions. In addition, due to its simplicity and low computational cost, the HEOM dissimilarity is a feasible choice as an optimization function in evolutionary algorithms.

In addition, we have also assumed that there is a fitness or optimization function to guide the search process of the metaheuristic algorithms. In this research, we explore the use of three different optimization functions to guide the search process. The selected functions are validity indexes that have been widely used to determine the clustering quality [29]. Two of them correspond to maximization functions, that is, the greater the value, the better the clustering and the later corresponds to a minimization function.

The silhouette is the average, for all clusters, of the silhouette width of the instances belonging to that cluster [29]. Let be $p$ an instance of the $C_l$ cluster. Its silhouette width is defined as $Sil\left(p\right)=\left[b\left(p\right)-a\left(p\right)\right]/\max \left\{a\left(p\right),b\left(p\right)\right\}$ where $a\left(p\right)$ is the average dissimilarity among $p$ and the other instances in its cluster, calculated as $a\left(p\right)=[1/\left|C_l\right|]{{\displaystyle \sum }}_{q\in C_l, p\ne q}diss\left(p,q\right)$ and $b\left(p\right)$ is the minimum of the average dissimilarities among $p$ and the instances belonging to every other clusters, and it is defined as $b\left(p\right)=\underset{h=1,\dots , k, h\ne l}{\min }\left\{[1/\left|C_h\right|]{\displaystyle \sum }_{q\in C_h}diss\left(p,q\right)\right\}$. The silhouette index of a set of clusters $C=\left\{C_1,\dots C_k\right\}$ is defined by:

$$Silhouette\left(C\right)=\frac{1}{k}{{\displaystyle \sum }}_{C_l\in C}\frac{1}{\left|C_l\right|}{\displaystyle \sum }_{p\in ]C_l}Sil\left(p\right)$$

(3)

For an instance $p$, its silhouette width is in the [−1, 1] interval. The greater the silhouette, the more compact and separated are the clusters.

The Dunn’s index for clustering considers the ratio between the minimum distance between two clusters, and the size of the bigger cluster. The same as the silhouette index, greater values correspond to better clustering [29].

$$D=\underset{1\le i\le k}{\min }\left\{\underset{\begin{array}{c}1\le j\le k\\ i\ne j\end{array}}{\min }\left\{\frac{diss\left(C_i,C_j\right)}{\underset{1\le \mathrm{l}\le \mathrm{k}}{\max }\left\{\mathsf{\Delta }\left(C_l\right)\right\}}\right\}\right\}$$

(4)

The Davies–Bouldin index is a well-known unsupervised cluster validity index [30]. It considers the dispersion of instances in a cluster $\Delta \left(C_i\right)$ and the dissimilarity of clusters $diss\left(C_i,C_j\right)$. The Davies–Bouldin index measures the average similarity between each cluster and its most similar cluster. The lower the values of the Davies–Bouldin index, the more compact and separated are the clusters. Formally, Davies–Bouldin index is defined as:

$$DB\left(C\right)=\frac{1}{k}{\displaystyle \sum }_{i=1‥k}\underset{j=1‥k, j\ne i}{\max }{R}_{i,j}$$

(5)

where $R_{i,j}$ is usually defined as $R_{i,j}=\frac{\Delta \left(C_i\right)+\Delta \left(C_j\right)}{diss(C_i, C_j)}$.

There are defined several inter-cluster dissimilarities, as well as measures of cluster size. In this research, we used the centroid dissimilarity, $diss\left(C_i,C_j\right)=diss\left(\overline{c_i},\overline{c_j}\right)$ as the inter-cluster measure, and the centroid measure $\Delta \left(C_i\right)={\displaystyle \sum }_{p\in C_i}diss\left(p,{\overline{c}}_i\right)/\left|C_i\right|$ for the cluster size.

4. Case Study

As a case study, to solve the clustering of mixed and incomplete data, we analyze three bio-inspired algorithms: the Artificial Bee Colony (ABC) algorithm [31], the Firefly Algorithm (FA) [32], and the Novel Bat Algorithm (NBA) [33], a recent development of the Bat Algorithm (BA) [34]. We selected these metaheuristic algorithms for four main reasons:

• All of them are metaheuristic algorithms, with a recent successful application to several optimization problems [35,36,37].
• They mimic different social behaviors, resulting in different approaches to explore the solution space.
• They have different approaches to exploit the best solutions, while avoiding trapping in local optima.
• ABC, FA, and BA algorithms have been successfully applied to clustering numerical data [5,11,12].

There are other bio-inspired algorithms fulfilling the above-mentioned reasons, such as Whale Optimization Algorithm [17], Dragonfly Algorithm [19], and others. However, we selected just three algorithms as a case of study, because the very large number of experiments needed to be performed for assessing their capabilities for clustering mixed and incomplete data. We suppose that the framework introduced in Section 3 is applicable to other bio-inspired algorithms as well.

In the later, we explain the selected metaheuristic algorithms, and we analyzed the common elements among them.

The Artificial Bee Colony (ABC) algorithm was introduced in 2007 [31], and it mimics the foraging of honey bees. The algorithm considers three kinds of bees: employed, onlooker, and scout bees. The solutions of the optimization problem are called food sources and have a position in the n-dimensional search space. Each food source has a nectar amount, corresponding to the desired fitness function of the optimization problem.

The ABC algorithm (Algorithm 2) start by sending the scout bees into the search space to randomly generate the initial food sources. Once obtained, the employed bees are assigned to the food sources. Then, each employed bee searches for a nearby food source in which the bee is employed. This searching for a new food source can be viewed as an updating mechanism, in which intervene the current solution, a randomly selected solution, and random numbers. If the new food source is better than the previous one, the employed bee discards the previous, and considers the new solution as its employed solution.

Algorithm 2. Pseudo code of the ABC algorithm.

ABC optimization algorithm

Inputs: fitness function of D dimensions; $I$: number of iterations; $\eta$: population size; $L$ limit of food sources

Output: $S_{best}=\left[s_{best,1},\dots s_{best,k}\right]$ best solution

Steps: Send a scout bee for random generation of $\eta$ food sources and select the best one as ${\mathrm{S}}_{\mathrm{best}}$ it = 1 While$it<I$ For each employed bee assigned to a food source $S_i$ Generate a new food source $S_{new}$, closer to the current source, with each component $s_{new,j}$ with $j\in \left[1,D\right]$, defined by $s_{new,j}=s_{i,j}+{\varphi }_{ij}\left(s_{i,j}- s_{k,j}\right)$, where ${\varphi }_{ij}$ is a random number in [−1, 1] and k is selected randomly in [1, $\eta$], with $k\ne i$. If $S_{new}$ is better than $S_i$ according to f, then $S_i\leftarrow S_{new}$; else increase limit of $S_i$ For each onlooker bee Fly to a food source with good nectar amount ($S_i$) with probability $p_i=\frac{f\left(S_i\right)}{{{\displaystyle \sum }}_{t=1}^{\eta }{S}_t}$ Generate a new food source $S_{new}$, closer to the selected source $S_i$, as in step 3.a.i. If $S_{new}$ is better than $S_i$, according to f, then $S_i\leftarrow S_{new}$; else increase limit of $S_i$ Update the best food source $S_{best}$ Send the scout bee to found the food sources that have reached the limit $L$, and replace them by randomly generated food sources it+ Return $S_{best}$

Based on the nectar amount of the food sources retained in the previous step, the ABC algorithm probabilistically determines which solutions the onlooker bees will visit. The onlooker bees visit those solutions and then fly around them to search for near food sources. Then, the algorithm considers a greedy selection process (same as the one carried out by the employed bees). After that, the scout bees search for exhausted food sources (not improved in $L$ iterations) and replace them by randomly generated new food sources. At the end of the pre-defined iterations, the ABC returns the best food source found.

The FA (Algorithm 3) was introduced in 2009 [32]. It mimics the coordinated flashing of fireflies. Although the real purpose of the bioluminescent behavior of fireflies is a current research topic for biologists, it is believed that it is related to finding mates, protecting against predators, and attracting potential preys. The FA algorithm consider some simplified rules about the behavior of fireflies: all are unisex, the attractive of a firefly is proportional to its brightness (the less attractive fireflies will move toward the most attractive fireflies) and the brightness of the fireflies is associated to a certain fitness function.

The attractiveness of fireflies is proportional to the intensity of the flashing seen by adjacent fireflies. The movement of a firefly $S_i$ being attracted by a firefly $S_j$ is determined by $S_i^{\prime }=S_i+\beta ·\left(S_j-S_i\right)+\alpha {\epsilon }_i$ where the second term of the expression is due to the attractive $\beta$, and the third term is a vector of random variables ${\epsilon }_i$ from a Gaussian distribution [32]. Best firefly moves randomly, that is, $S_{best}^{\prime }=S_{best}+\alpha {\epsilon }_i$.

Algorithm 3. Pseudo code of the FA algorithm.

FA optimization algorithm

Inputs: fitness function of D dimensions; $I$: number of iterations; $\eta$: population size;

Output: $S_{best}=\left[s_{best,1},\dots s_{best,k}\right]$ best solution

Steps: Randomly generate $\eta$ fireflies and select the best one as $S_{best}$ it = 1 While$it<I$ For each firefly $S_i$ For each firefly $S_j$ If $S_j$ is better than $S_i$, then update $S_i$ by modifying each component as $s_{ik}=s_{ik}+\beta \left(s_{jk}-s_{ik}\right)+\alpha {\epsilon }_i$, with $k\in \left[1,D\right]$ Select the best firefly $S_{best}$, and update it by modifying each component as $s_{best,k}=s_{ik}+\beta \left(s_{jk}-s_{ik}\right)+\alpha {\epsilon }_i$ it+ Return $S_{best}$

In 2009, Lukasik y Zak [38] experimentally obtained the optimum values for the parameters of the FA algorithm. They conclude that the best values of the parameters are $\alpha =0.01$, $\beta =1$ and the population number $\eta$ varying among 15 and 50 fireflies. These conclusions allow us to simplify the movement of a firefly, as follows: $S_i^{\prime }=S_i+\beta ·\left(S_j-S_i\right)+\alpha {\epsilon }_i$, with $\alpha =0.01$ and $\beta =1$.

Thus, the movement of the firefly becomes $S_i^{\prime }=S_i+S_j-S_i+0.01{\epsilon }_i=S_j+0.01{\epsilon }_i$. That is, the movement of a firefly $S_i$ being attracted by a firefly $S_j$ is determined by just a modification (or update) of the position of the $S_j$ firefly.

The Novel Bat Algorithm (NBA) was proposed in 2015 [33] as an extension of the Bat Algorithm [34]. The NBA incorporates the compensation of the Doppler Effect in echoes by the bats. In addition, in NBA, the bats can forage in different habitats [33]. NBA uses a probability-based approach to determine the habitat selection of the bats. The bats use either a quantum-based approach, or a mechanical approach for habitat selection. In the quantum approach, the bats consider their positions, the position of the best bat and the mean position of the swarm to obtain new solutions. On the contrary, in the mechanical approach, the bats compensate the Doppler Effect and use the information in the updating process. Finally, the bats perform a local search based on the position of the best bat. The parameters of the NBA algorithm are updated with each iteration. Algorithm 4 shows the main steps of the NBA algorithm.

Although different, the analyzed algorithms have two things in common: They start by randomly generating the solutions, and then they modify the current solutions by considering a single solution (as in FA) or a combination of the current solution with another solution (as in ABC and NBA).

In either case, a random variation is introduced. Thus, the analyzed bio-inspired algorithms can be considered to operate in three stages: initialization, updating and returning. The updating stage, although different in every algorithm, includes a mechanism to modify or to update the current solutions. We used this common behavior to provide a unified representation of the solutions, and a strategy to update the clustering solutions in the optimization process.

We consider that the proposed framework for solution representation and updating is applicable to several other bio-inspired algorithms.

In the following, we explain the proposed integration into the three analyzed swarm intelligence algorithms. We introduce three clustering algorithms: The clustering based on Artificial Bee Colony (CABC), the clustering based on Firefly Algorithm (CFA), and the clustering based on Novel Bat Algorithm (CNBA). For the CABC algorithm, the pseudo code of the clustering approach is given in Algorithm 5. Note that the ABC original idea is preserved but contextualized to the mixed data clustering problem. On the other hand, for the CFA algorithm, we include the updating strategy as shown in Algorithm 6, to obtain the desired clustering.

Finally, for the CNBA algorithm, which has a more complex structure, we include the updating strategy as in Algorithm 7. In each case, we considered the elements that intervene in the updating of a certain solution, and integrate then in the proposed modification strategy, by considering the best solutions so far, the current solution, and a random process. Our approach, although quite simple, is very effective in modelling the mixed data clustering as optimization problem. In addition, we consider that this approach is useful, and can be effortless applied to other swarm intelligence algorithms.

Algorithm 4. Pseudo code of the NBA algorithm.

NBA optimization algorithm

Inputs: f: fitness function of D dimensions; $I$: number of iterations; $\eta$: population size; $prob$: probability for habitat selection; $pe{r}_i$: pulse emission rate of the i-th bat, w: inertia weight; Dopp: the compensation rates for Doppler effect in echoes; h: contraction–expansion coefficient; G: the frequency of updating the loudness and pulse emission rate; $\alpha ,\gamma ,f_{min},f_{max},A_0,r_0$: parameters in basic Bat Algorithm

Output: $S_{best}=\left[s_{best,1},\dots s_{best,k}\right]$ best solution

Steps: Randomly generate $\eta$ bats and select the best ones as $S_{best}$ Randomly generate the velocities ${\upsilon }_i$ for each bat $S_i$ it = 1 While$it<I$ For each bat $S_i$ If $rdm\left(0,1\right)\le prob$ Generate a new bat $S_{new}$, closer to the best bat $S_{best}$, with each component defined as follows, where $u_{ij}$ is a number uniformly distributed between 0 and 1, and greater than zero; and $mea{n}_j$ is the mean value of the dimension j in all bats. $$s_{new,j}=\left\{\begin{array}{cc}{s}_{best,j}+\theta \ast \left|mea{n}_j-s_{i,j}\right|\ast ln\left(\frac{1}{u_{ij}}\right)& if rdm\le 0.5\\ s_{best,j}-\theta \ast \left|mea{n}_j-s_{i,j}\right|\ast ln\left(\frac{1}{u_{ij}}\right)& otherwise\end{array}\right.$$ Else Generate a new bat $S_{new}$, closer to the current bat $S_i$ Update the velocity components of the bat ${\upsilon }_{i,j}$ as ${\upsilon }_{i,j}=w\ast {\upsilon }_{i,j}+{f_{ij}}^{\prime }\left(s_{best,j}-s_{i,j}\right)$ where ${f_{ij}}^{\prime }=\frac{c+{\upsilon }_{i,j}}{c+{\upsilon }_{best,j}}\ast f_{ij}\ast \left(1+Dop{p}_i\ast \frac{s_{best,j}-s_{i,j}}{\left|s_{best,j}-s_{i,j}\right|+\epsilon }\right)$. Note: c = 340 m/s is a constant. Update the bat components as $s_{new,j}=s_{i,j}+{\upsilon }_{i,j}$ If ($rdm\left(0,1\right)\le pe{r}_i$) Generate a new bat $S_{new}$, closer to the best bat, with component defined as $s_{new,j}=s_{best,j}\ast \left(1+\zeta \right)$ where $\zeta$ is a random number drawn from a Gaussian distribution with mean zero and standar deviation ${\sigma }^2=\left|A_i-A_{mean}\right|+\epsilon$. $A_{mean}$ is the average loudness of all bats. Evaluate the fitness of the newly generated bats Update pulse emission rates and other parameters Update $S_{best}$. If the best solution is not updated in G iterations, re-initialize the loudness and set temporary pulse rates it+ Return $S_{best}$

Algorithm 5. Pseudo code of the CABC algorithm to clustering mixed data.

CABC mixed clustering algorithm

Inputs: $X$: dataset of instances; $k$: cluster number; $I$: number of iterations; $\eta$: population size; $L$ limit of food sources

Output: $C=\left\{C_1,\dots C_k\right\}$: clustering of instances

Steps: 5. Send a scout bee for random generation of $\eta$ food sources 6. it = 1 7. While$it<I$ For each employed bee assigned to a food source $S_i$ Generate a new food source $S_{new}$, closer to the current source, as Updating($S_i$) If $S_{new}$ is better than $S_i$, then $S_i\leftarrow S_{new}$; else increase limit of $S_i$ For each onlooker bee Fly to a food source with good nectar amount ($S_i$) Generate a new food source $S_{new}$, closer to the current source, as Updating($S_{new}$) If $S_{new}$ is better than $S_i$, then $S_i\leftarrow S_{new}$; else increase limit of $S_i$ Send the scout bee to found the food sources that have reached the limit $L$, and replace them by randomly generated food sources it+ 8. Create a cluster $C$ by assigning the instances in $X$ to its closest centers, considering the centers of the best food source 9. Return $C$

Algorithm 6. Pseudo code of the CFA algorithm to clustering mixed data.

CFA mixed clustering algorithm

Inputs: $X$: dataset of instances; $k$: cluster number; $I$: number of iterations; $\eta$: population size

Output: $C=\left\{C_1,\dots C_k\right\}$: clustering of instances

Steps: Randomly generate $\eta$ fireflies it = 1 While$it<I$ For each firefly $S_i$ For each firefly $S_j$ If $S_j$ is better than $S_i$, then $S_i$ = Updating($S_j$) Select the best firefly $S_{best}$ $S_{best}$= Updating($S_{best}$) it+ Create a cluster $C$ by assigning the instances in $X$ to its closest centers, considering the centers of the best firefly Return $C$

Algorithm 7. Pseudo code of the CNBA algorithm to clustering mixed data.

CNBA mixed clustering algorithm

Inputs: $X$: dataset of instances; $k$: cluster number; $I$: number of iterations; $\eta$: population size; $prob$: probability for habitat selection; $pe{r}_i$: pulse emission rate of the i-th bat

Output: $C=\left\{C_1,\dots C_k\right\}$: clustering of instances

Steps: 6. Randomly generate $\eta$ bats and select the best one $S_{best}$ 7. it = 1 8. While$it<I$ For each bat $S_i$ If $rdm\left(0,1\right)\le prob$ If $rdm\left(0,1\right)\le 0.5$ Generate a new bat $S_{new}$, closer to the best bat, as Updating($S_{best}$) Else Generate a new bat $S_{new}$, closer to a random bat, as Updating($S_{rdm\left(1,\eta \right)}$) Else Generate a new bat $S_{new}$, closer to the current bat, as Updating($S_i$) If ($rdm\left(0,1\right)\le pe{r}_i$) Generate a new bat $S_{new}$, closer to the best bat, as Updating($S_{best}$) Evaluate the fitness of the newly generated bats and update $S_{best}$ Update pulse emission rates and other parameters it+ 9. Create a cluster $C$ by assigning the instances in $X$ to its closest centers, considering the centers of the best bat 10. Return $C$

5. Results and Discussion

In this section we explain the experiments made, as well as the used datasets, performance measures and statistical analysis, to determine the performance of the proposed clustering framework for mixed and incomplete data, by using Swarm Intelligence.

5.1. Datasets and Cluster Definition

To compare the proposed algorithm for clustering mixed and incomplete data, we used 15 mixed and/or incomplete datasets from the University of California at Irvine (UCI) repository of Machine Learning [17].

Table 1. Description of the datasets used in the numerical experiments.

Dataset	Instances	Features		Classes	Missing
		Numerical	Categorical
anneal	898	6	32	5	x
autos	205	15	10	6	x
cmc	1473	2	8	3	x
colic.ORIG	368	7	21	3	x
credit-a	690	6	9	2	x
credit-g	1000	7	13	2
dermatology	366	1	33	6	x
heart-c	303	6	7	2	x
hepatitis	155	6	13	2	x
labor	57	8	8	2	x
lymph	148	3	15	4
postoperative	90	0	9	3	x
tae	151	3	3	2	x
vowel	990	10	3	11
zoo	101	1	16	7

gives the description of the considered datasets.

All the selected datasets have labelled instances; thus, it is possible to compare the results obtained by the clustering algorithms with respect to the labels of the data. Although in several studies the cluster number is greater than the number of classes (ex. having $k=50$ clusters, and only two or three classes [21]), we want to explore the use of bioinspired algorithms for finding the natural structure of data.

We are considering in the experiments that the natural structure of data is represented by the class labels. Thus, in our experiments, the instances of the same class should belong to the same cluster. We are not dealing with the issue of possible mislabeled or noisy instances. That is the reason why, in our experiments, the number of clusters to obtain was set as the number of classes in the corresponding dataset (column “Classes” of Table 1). Thus, our cluster number is in the $\left\{2,3,4,5,6,7,11\right\}$ set.

5.2. Performance Metrics and Statistical Tests

For comparing the results obtained by the algorithms, we used the cluster error measure [29]. Cluster error is one of the most used validity indexes. It matches the clusters obtained by the clustering algorithm with the clusters in the ground truth clustering.

Then, it counts the instances whose cluster assignment differs. The cluster error measure is the opposite of the purity measure, who counts the instances having equal cluster assignment in $GT$ and $C$. The lower the cluster error, the better the clustering. Let $GT=\left\{g{t}_1,\dots ,g{t}_k\right\}$ be the ground truth clustering, $C$ the obtained clustering and $\left|C_i\right|$ the number of instances in cluster $C_i\in C$. Let $n$ be the total number of instances, $e_i^j$ the number of instances in the cluster $C_i\in C$, not belonging to the cluster $g{t}_j\in GT$. The cluster error is given by:

$$CE\left(C\right)= {\displaystyle \sum }_{C_iϵC}\frac{1}{n}\underset{g{t}_jϵGT}{\min }\left\{e_i^j\right\}$$

(6)

We also use the adjusted Rand Iindex (ARI) [39] as the cluster performance measure. ARI is based on the computation of a contingency table between the real and obtained data partitions. Let us have $n$ instances, and two partitions, $GT$ and $C$. The partition $GT$ has $k$ clusters, and the partition $C$ has $s$ clusters. We first compute a contingency table as in Figure 4, where the cells $N_{ij}$ with $1\le i\le k; 1\le j\le s$ represent the number of instances in both $g{t}_i$ and $C_j$ clusters.

Then, we set three auxiliary variables x, y and z, to compute the ARI as $x= {\displaystyle \sum }_{i,j}\left(\begin{array}{c}{N}_{ij}\\ 2\end{array}\right)$, $y= {\displaystyle \sum }_i\left(\begin{array}{c}{a}_i\\ 2\end{array}\right)$, and $z= {\displaystyle \sum }_j\left(\begin{array}{c}{b}_j\\ 2\end{array}\right)$, and the ARI is computed as follows:

$$ARI= \frac{x-\left[\frac{y\ast z}{\left(\begin{array}{c}n\\ 2\end{array}\right)}\right]}{\frac{1}{2}\left[y+z\right]-\left[\frac{y\ast z}{\left(\begin{array}{c}n\\ 2\end{array}\right)}\right]}$$

(7)

For the statistical comparison of results, we used the Wilcoxon signed-rank test to compare pair of algorithms. This is a non-parametric test for comparing paired differences among related samples. This test has been recommended by Demsar [40] for comparing pairs of algorithms over multiple datasets.

We use the Wilcoxon signed-rank test to determine the existence of significant differences in the clustering obtained by the different algorithms. We used the IBM SPSS Statistics 20 for computing this test [41].

We also used the Friedman test for related samples [25,35], followed by the Holm’s post-hoc test [36], to compare multiple samples over multiple datasets. Both tests are recommended in [15]. We used the KEEL software [24] for computing both tests. Our samples are related, due to the clustering algorithms are applied over the same datasets.

For all statistical tests, we set the null hypothesis as no differences, and the alternative hypothesis considering differences in performance. We use a significance level of 0.05, for a 95% level of confidence.

5.3. Experimental Design

For all experiments, we evaluated ten independent executions of the algorithms and averaged the results. We reported in the Supplementary Data Files the average and standard deviation obtained by the algorithms in all experiments according to cluster error and adjusted Rand index (ARI), as well as the results of the application of the statistical tests for cluster error. In this section, we will report the results of the Friedman and Holm tests for ARI measure, and the numerical results for both measures at Section 5.3.3.

5.3.1. Parameter configuration

First, to tune the main parameters of the metaheuristic algorithms, we carried out two experiments. The Experiment #1 was to determine the adequate iteration number of the algorithms, considering a small population (only 10 individuals). We evaluated the proposed algorithms considering 50, 100, 500, and 1000 iterations. We used the Davies–Bouldin index as the fitness function. We report in

Table 2. Ranking of Friedman test comparing the results of the proposed algorithms with different iteration number, according to ARI measure.

Algorithm	Iterations	Ranking	p-Value
CABC	100	1.7333	0.027238
	50	2.4667
	500	2.6667
	1000	3.1333
CFA	500	2.1333	0.430127
	50	2.3333
	1000	2.7333
	100	2.8000
CNBA	50	2.0000	0.039129
	500	2.2000
	1000	2.5333
	100	3.2667

and

Table 3. Results of the Holm test comparing the best performed iteration number of each algorithm with respect other iteration number, according to ARI measure.

Algorithm	i	Iterations	z	p-Value	Holm $(\mathit{\alpha }/\mathit{i})$
CABC	3	1000	2.969848	0.002979	0.016667
	2	500	1.979899	0.047715	0.025000
	1	50	1.555635	0.119795	0.05000
CNBA	3	100	2.687006	0.00721	0.016667
	2	1000	1.131371	0.257899	0.025000
	1	500	0.424264	0.671373	0.050000

the results of the Friedman test ranking and Holm’s post-hoc tests, respectively.

The Friedman test rejected the null hypothesis for algorithms CABC and CNBA, therefore, we applied the Holm test (Table 3). Holm’s procedure rejects those hypotheses that have an unadjusted p-value lower or equal than 0:025, for both algorithms.

For CABC, Holm’s test does not reject the null hypothesis while comparing the results of using 100 iterations with the results of using 50 and 500 iterations. However, the test rejects the null hypothesis with respect using 1000 iterations. Considering the results of ARI, for using 100 and 1000 iterations (Supplementary Table S4), we can conclude that using 1000 iterations is worse than using 100. In our opinion, this is due to the CABC algorithm is overtrained.

For CNBA, the Holm’s test does not reject the null hypothesis while comparing the results of using 50 iterations with the results of using 500 and 1000 iterations. However, the test rejects the null hypothesis with respect using 100 iterations. Considering the results of ARI, for using 50 and 100 iterations (Supplementary Table S6), we can conclude that using 50 iterations is better than using 100. This result can be due to CNBA algorithm can lose good solutions and replace them by worse solutions (as does NBA), showing this good initial (50 iterations) and final (500 and 1000 iterations) behavior, with a gap of suboptimal performance at 100 iterations.

Due to the above results, we conclude than 50 iterations were sufficient to obtain high quality clusters.

Experiment #2 was to determine the adequate population size for the proposed algorithms. In this experiment, we evaluated populations of 10, 20, 30, and 40 individuals. Again, we used the the Davies–Bouldin index as fitness function. We want to test the influence of the population size in the performance of the algorithms, due to swarm intelligence algorithms are sensible to changes in population size [6]. Therefore, establishing an adequate population number for clustering will help the user to select good parameters for the proposed algorithms. In

Table 4. Ranking of Friedman test comparing the results of the proposed algorithms with different population size, according to the ARI measure.

Algorithm	Population	Ranking	p-Value
CABC	40	2.2667	0.686894
	10	2.3333
	30	2.6667
	20	2.7333
CFA	30	2.3333	0.781904
	40	2.3333
	10	2.6000
	20	2.7333
CNBA	10	2.2667	0.762613
	40	2.4000
	30	2.6000
	20	2.7333

we report the results of the statistical tests made to assess the existence or not of significant differences in performance of the three proposed algorithms, while using different population sizes.

As shown, the Friedman test did not reject the null hypothesis for any of the compared algorithms, and no significant differences were found regarding the use of different population sizes. As shown, increasing the population number did not improve the results of the clustering process. The above results show the capabilities of our proposal, which obtained good clustering with only ten individuals and 50 iterations. These numbers are quite low and allows applying the algorithms over large datasets. In addition, we can conclude that those parameter values are good enough to obtain high-quality clusters.

5.3.2. Influence of the Fitness Function in the Clustering Results

The choice of an adequate fitness function is crucial for metaheuristic algorithms, due to the fitness values guide the optimization processes. We tested the influence of the fitness function in the results obtained by the proposed algorithms for clustering mixed and incomplete data. We compared the results using the Davies–Bouldin index, the Dunn´s index and the Silhouette index as fitness functions.

In this experiment, we again evaluated the clustering results considering cluster error and adjusted Rand index. In the Supplementary Data, we show the results of the algorithms considering these functions.

For the cluster error measure, the Friedman tests obtained p-values of 0.361799, 0.627089, and 0.165299 for CABC, CFA and CNBA, respectively. The tests did not find significant differences in the performance of the proposed algorithms, while using different fitness functions. The tests support the hypothesis of similar performance disregarding the validity index used by our modification of the bio-inspired algorithms for mixed and incomplete data clustering. These results confirm that the proposed algorithms are robust and do not heavily depend on the optimization function used by the optimization procedure, while using cluster error measure.

For the adjusted Rand index measure, the Friedman tests (

Table 5. Ranking of Friedman test comparing the results of the proposed algorithms with different fitness functions, according to ARI measure.

Algorithm	Fitness Function	Ranking	p-Value
CABC	Dunn	1.6667	0.038133
	Silhouette	1.8000
	Davies–Bouldin	2.5333
CFA	Dunn	1.8667	0.449329
	Silhouette	1.8667
	Davies–Bouldin	2.2667
CNBA	Silhouette	1.7333	0.154638
	Dunn	1.8667
	Davies–Bouldin	2.4000

) also support the hypothesis of equal performance disregarding the fitness function used, for both CFA and CNBA algorithms. However, for the CABC algorithm, the test rejects the null hypothesis and we applied the Holm’s test (

Table 6. Results of the Holm test comparing the best performed fitness function of each algorithm with respect other fitness functions, according to the ARI measure.

Algorithm	I	Function	z	p-Value	Holm $(\mathit{\alpha }/\mathit{i})$
CABC	2	Davies–Bouldin	2.373464	0.017622	0.025
	1	Silhouette	0.365148	0.715001	0.050

).

The Holm’s procedure rejects those hypotheses that have an unadjusted p-value lower or equal than 0:05. That is, the test rejects the null hypothesis of equal performance while using Davies–Bouldin as fitness function, with respect using Dunn’s index. Considering the results of the CABC using both functions (Supplementary Table S18), the use of Dunn’s index is better than using Davies–Bouldin as the fitness function, with respect to the adjusted Rand index measure.

5.3.3. Comparison with Respect to Other Clustering Algorithms

At this point, we compare the proposed algorithm for clustering mixed and incomplete data with respect to previously reported algorithms. We used the algorithms k-Prototypes [22], AGKA [4] and AD2011 [21] in the comparisons. All of them deal with mixed data. The k-Prototypes algorithm has a specific dissimilarity function, which separates the computation of numeric and categorical feature dissimilarities, as well as missing feature values. The AGKA uses a dissimilarity function proposed by Ahmad and Dey in 2007 [42]. Such dissimilarity considers separately numeric and categorical features, and accounts for categorical feature value dissimilarities, in a personalized way. The same authors (Ahmad and Dey) refined their 2007 algorithm, and the result is the AD2011 algorithm.

For handling missing values directly, we modified the k-Prototypes algorithm in the following:

(a)

We did not consider missing values in mean nor in mode computations.

(b)

For numerical missing values, we replace the missing value by the mean value in the dissimilarity computation.

(c)

For categorical missing values, we consider them to be different to every other (non-missing) value, for the dissimilarity computation, and equal to other missing values.

For the AD2011 and AGKA algorithms, which use the same dissimilarity function and the same procedure for computing cluster centers, we modify the algorithms to make them able to deal with missing values, in the following:

(a)

We did not consider numerical missing values for the discretization procedure.

(b)

For numerical missing values, we replace the missing value by the mean value in the dissimilarity computation.

(c)

For categorical attributes, we consider the value “missing” as another admissible value in the procedures for finding significance and for computing cluster centers.

As performance measures, we considered cluster error and adjusted Rand index. The proposed algorithms used the Dunn’s index as fitness function, as well as ten individuals and 50 iterations.

Table 7. Performance results according to cluster error of CABC, CFA and CNBA with respect to other algorithms.

Dataset	CABC	CFA	CNBA	kPrototypes	AGKA	AD2011
anneal	0.21	0.21	0.19	0.24	0.26	0.24
autos	0.45	0.51	0.49	0.59	0.62	0.67
cmc	0.57	0.57	0.57	0.57	0.58	0.57
colic.ORIG	0.34	0.37	0.36	0.37	0.37	0.37
credit-a	0.34	0.35	0.25	0.44	0.43	0.44
credit-g	0.29	0.3	0.3	0.3	0.29	0.3
dermatology	0.19	0.25	0.24	0.69	0.73	0.69
heart-c	0.18	0.19	0.17	0.42	0.47	0.46
hepatitis	0.18	0.18	0.18	0.21	0.21	0.21
labor	0.28	0.33	0.30	0.35	0.37	0.35
lymph	0.24	0.26	0.27	0.32	0.42	0.45
postoperative	0.33	0.36	0.32	0.30	0.34	0.30
tae	0.54	0.57	0.53	0.58	0.62	0.66
vowel	0.84	0.84	0.84	0.68	0.84	0.91
zoo	0.07	0.04	0.08	0.21	0.54	0.59

show the obtained results for cluster error, and the best results are highlighted in Supplementary Data Files.

As shown in Table 7, the proposed algorithm for clustering mixed and incomplete data clearly outperformed previously reported algorithms, but for datasets postoperative and vowel.

Table 8. Performance results according to adjusted Rand index of CABC, CFA and CNBA with respect to other algorithms.

Dataset	CABC	CFA	CNBA	kPrototypes	AGKA	AD2011
anneal	0.06	0.02	0.05	0.00	0.01	0.00
autos	0.08	0.08	0.10	0.04	0.01	0.00
cmc	0.01	0.00	0.00	0.01	0.01	0.00
colic.ORIG	0.06	0.11	0.07	−0.03	−0.01	0.00
credit-a	0.06	0.01	0.11	0.01	0.00	0.00
credit-g	0.02	0.01	0.01	0.03	0.01	0.00
dermatology	0.27	0.19	0.23	−0.01	0.00	0.00
heart-c	0.10	0.06	0.10	0.02	0.00	0.00
hepatitis	0.01	0.01	0.02	0.04	0.00	0.00
labor	0.04	0.13	0.03	−0.05	0.00	0.00
lymph	0.09	0.06	0.07	0.14	0.02	0.00
postoperative	0.01	0.02	0.02	−0.01	0.02	0.00
tae	0.01	0.04	0.02	0.01	0.00	0.00
vowel	0.02	0.03	0.03	0.16	0.01	0.00
zoo	0.61	0.52	0.52	0.45	0.05	0.00

show the obtained results for adjusted Rand index.

To establish if the differences in performance were significant or not, we again applied non-parametric statistical tests. The results for the Friedman tests are shown in

Table 9. Ranking of Friedman test comparing the results of algorithms, according to cluster error and ARI measures.

Measure	Algorithm	Ranking	p-Value
Cluster Error	CABC	1.9667	0.000003
	CNBA	2.2667
	CFA	3.1333
	kPrototypes	3.9000
	AD2011	4.8333
	AGKA	4.9000
Adjusted Rand Index	CABC	2.2667	0.000003
	CNBA	2.5333
	CFA	2.6667
	kPrototypes	3.6667
	AGKA	4.4000
	AD2011	5.4667

, while the ones for Holm’s tests are shown in

Table 10. Results of the Holm test comparing the best performed algorithm with respect others, according to the cluster error and ARI measures.

Algorithm	i	Algorithms	z	p-Value	$\mathbf{Holm} (\mathit{\alpha }/\mathit{i})$
Cluster Error	5	AGKA	4.29396	0.000018	0.010000
	4	AD2011	4.19637	0.000027	0.012500
	3	kPrototypes	2.83011	0.004653	0.016667
	2	CFA	1.707825	0.087669	0.025000
	1	CNBA	0.439155	0.660549	0.050000
Adjusted Rand Index	5	AD2011	4.68432	0.000003	0.010000
	4	AGKA	3.12288	0.001791	0.012500
	3	kPrototypes	2.04939	0.040424	0.016667
	2	CFA	0.58554	0.558185	0.025000
	1	CNBA	0.39036	0.696270	0.050000

.

As shown, Friedman tests reject the null hypothesis for both the cluster error and ARI measures, evidencing the existence of significant differences in the performance of the algorithms. For the cluster error measure, Holm’s procedure rejects those hypotheses that have an unadjusted p-value lower or equal than 0.025, and for ARI, the ones with an unadjusted p-value lower or equal than 0.016667. Therefore, and analyzing the results of Table 7 and Table 8, we can conclude that the proposed CABC algorithm is significantly better than k-Prototypes, AGKA and AD2011 for the cluster error, and better than AGKA and AD2011 for ARI measure.

These numerical experiments allow us to conclude that the proposed strategy for clustering mixed and incomplete data using metaheuristic algorithms is highly competitive and leads to better results than other clustering algorithms.

As result of our research, we propose a representation of the clusters, which selects as cluster centers instances of the dataset. This is the first contribution to handle mixed and incomplete data. We did not create artificial instances by averaging numerical feature values and using the mode for categorical features; instead, we consider feature dependencies by selecting existing instances of the dataset. Second, we update the cluster centers by an updating procedure which changes the cluster center by another existing instance, instead of creating and artificial instance to be the center. In addition, we make our proposal able to be used with any dissimilarity function (HEOM is just an example used for the experiments), which allows the user to cluster the data with any function of their choice. This is a fundamental difference of previous proposals, which use fixed dissimilarity functions.

6. Conclusions

We have used several swarm intelligence algorithms to solve mixed and incomplete data clustering. Our proposal is emerging as a valuable contribution to the field, because most of the proposed algorithms for clustering operate only over numerical data, and therefore these algorithms are not capable of solving problems that are relevant for human activities and whose data are described not only by numeric attributes, but by mixed and incomplete data. Our proposal becomes a response to a problematic situation related to the fact that the methods and algorithm for clustering over mixed and incomplete data, are an evident minority in scientific literature, in addition to being ineffective.

In order to achieve an effective proposal that solves these types of problems, we have introduced a generic modification to three swarm intelligence algorithms (Artificial Bee Colony, Firefly Algorithm, and Novel Bat Algorithm). We also provide an unbiased comparison among several metaheuristics based clustering algorithms, concluding that the clusters obtained by our proposals are highly representative of the “natural structure” of data.

Our proposal significantly outperformed the k-Prototypes, AGKA and AD2011 clustering algorithms according to cluster error, and AGKA and AD2011 according to the adjusted Rand index. The numerical experiment allows us to conclude that the proposed strategy for clustering mixed and incomplete data using metaheuristic algorithms is highly competitive and leads to better results than other clustering algorithms. In addition, the experiments showed that the proposed algorithms are robust and do not heavily depend on the optimization function used. However, for CABC, we find that using Dunn’s index as fitness function led to better results according to the adjusted Rand index.

In addition, after a thoughtful analysis and a very large number of experiments, the statistical tests shown several useful results about the adequate parameter configuration for some metaheuristic algorithms applied to clustering mixed and incomplete data. Thus, according to the Wilcoxon test, we cannot reject the hypothesis of equality of performance between 10 individuals and greater number of individuals (20, 40, and 50); in addition, the test cannot reject the hypothesis of equality of performance between using 50 iterations and great number of iterations (100, 500, and 1000), in clustering mixed and incomplete data.