An Improved Three-Way K-Means Algorithm by Optimizing Cluster Centers

Abstract

Most of data set can be represented in an asymmetric matrix. How to mine the uncertain information from the matrix is the primary task of data processing. As a typical unsupervised learning method, three-way k-means clustering algorithm uses core region and fringe region to represent clusters, which can effectively deal with the problem of inaccurate decision-making caused by inaccurate information or insufficient data. However, same with k-means algorithm, three-way k-means also has the problems that the clustering results are dependent on the random selection of clustering centers and easy to fall into the problem of local optimization. In order to solve this problem, this paper presents an improved three-way k-means algorithm by integrating ant colony algorithm and three-way k-means. Through using the random probability selection strategy and the positive and negative feedback mechanism of pheromone in ant colony algorithm, the sensitivity of the three k-means clustering algorithms to the initial clustering center is optimized through continuous updating iterations, so as to avoid the clustering results easily falling into local optimization. Dynamically adjust the weights of the core domain and the boundary domain to avoid the influence of artificially set parameters on the clustering results. The experiments on UCI data sets show that the proposed algorithm can improve the performances of three-way k-means clustering results and is effective in revealing cluster structures.

1. Introduction

Rapid developments of science and technology produce a large number of data anytime and anywhere in modern society. How to mine valuable information from massive data has been a challenging task of information science and artificial intelligence. As one emerging technology of intelligence information processing, granular computing [1,2] deals with information in the form of some aggregates and their processing. The main task of granular computing is to construct different granular structures by various methods of information granulation [3]. There are many different approaches of information granulation, among which, clustering analysis is one of the widely and most used one [4]. Clustering analysis [5] is a multivariate analysis method in statistics. The objective of clustering is to divide a set of objects into different groups such that the objects in the same cluster have high similarity while the objects in the different groups have high dissimilarity. As a powerful data analysis technique, clustering has been widely used information granulation [6,7,8], information fusion [9,10,11], attribute reduction [12,13,14,15], feature selection [16,17,18], and other fields.

There are many different clustering algorithms, among which, k-means [19] is one of the most commonly used one. The object of k-means algorithm is to minimize the following function,

$$J=min\sum _{j=1}^k\sum _{v_i\in C_j}\left|\right|v_i-{\mu }_j{\left|\right|}^2,$$

(1)

where $\left|\right|v_i-{\mu }_j\left|\right|$ is a chosen distance measure between a data point $v_i$ and the cluster centre ${\mu }_j$ of j-th cluster. The process of k-means has three steps. The first one is to randomly select k samples as the initial cluster centers, the second one is to calculate the distance between each sample and cluster center, and the third one is to assign the samples to the nearest cluster. The process is repeated by continuously updating the iterative cluster centers until a certain termination condition is met. Although k-means algorithm has been widely used since it was proposed, it still has the following problems.

• The clustering results of k-means are dependent on the random selection of clustering centers and the problem of local optimization readily occurs.
• Traditional k-means algorithms are based on the assumption that a cluster is represented by one single set with a sharp boundary. Only two types of relationship between an object and cluster are considered, i.e., belong to and not belong to. The requirement for a sharp boundary is easily met for analytical clustering results, but may not adequately show the uncertainty information in the dataset.

To solve the above problems, many methods have been developed to improve the results of the k-means algorithm. For example, Zhang et al. [20] proposed an improved k-means algorithm based on a density canopy to solve the problem of determining the best initial seeds. Wang et al. [21] presented a three-way k-means algorithm by integrating three-way decisions [22,23] into clustering to depict the uncertainty information in the dataset. The main idea of the three-way k-means algorithm is to introduce fault-tolerant errors in the k-means iteration process, and represents the results of each cluster with a core region and a fringe region. However, in common with the k-means algorithm, the three-way k-means algorithm also has the problems that the clustering results are dependent on the random selection of clustering centers, are sensitive to noise points and outliers, and easily succumb to local optimization.

To solve the local optimal problem of the three-way k-means algorithm, this paper presents an improved three-way k-means algorithm by integrating the ant colony [24] algorithm into the k-means algorithm. The ant colony algorithm simulates the foraging behavior of ants in nature, and initially solved a problem of traveling salesmen by analogy with a pheromone mechanism. Its inspiration came from the behavior of ants in finding the shortest path in the process of looking for food. Compared with other clustering algorithms, the ant colony algorithm has the advantages of strong robustness, good parallelism, adaptability to computer systems, ease of combination with other algorithms and others.

The positive pheromone feedback mechanism in the ant colony algorithm can be transformed into a clustering problem. In the ant colony clustering algorithm, based on ant colony foraging behavior, food sources are regarded as different clustering centers, and data are regarded as ants with different attributes. Under the guidance of pheromones, ants move between different food sources with a certain probability, and finally form clustering results around different food sources. In the proposed algorithm, we use the random probability selection strategy of ant decision-making in the ant colony algorithm to optimize the initial clustering centers. On this basis, iterative updating can better solve the problem of the sensitivity of the three-way k-means algorithm to the initial clustering center.

This article is structured in the following manner: In Section 2, we introduce the background to our proposed methods, including three-way clustering and three-way k-means. The detailed process of the proposed algorithm is presented in Section 3. In Section 4, experiments are described which evaluate the performance of the proposed algorithms. Some concluding remarks and discussion of future research directions are presented in Section 5.

2. Related Work

2.1. Three-Way Clustering

In classical two-way decision clustering, a cluster is represented by a set with a crisp boundary. There are only two relationships between an object and a class: the object either belongs or does not belong to the class. The requirement of a crisp boundary is conducive to analysis results, but may not adequately adequately show the uncertainty information in the data set. In order to address the problem of information uncertainty, Yao [22,23] proposed a theory of three-way decision by extending the commonly used binary-decision model. There are three main tasks in three-way decision [25]: (1) trisecting strategies, (2) acting strategies, and (3) outcome evaluation. The whole process can be depicted as TAO (trisecting-acting-outcome) [25] framework shown in Figure 1.

With the development of three-way decision, many applications of three-way decision were researched in various fields, such as, data analysis [26,27], concept analysis [28,29,30], granular computing [31,32], sequential three-way decision [33,34].

To relax this requirement of sharp boundary, a new type of clustering algorithms [35,36] was proposed, named three-way clustering. Unlike traditional hard clustering, three-way clustering presents a cluster through a pair of sets:

$$C_i=(Co\left(C_i\right),Fr\left(C_i\right)),$$

(2)

where $Co\left(C_i\right)\subset V$ and $Fr\left(C_i\right)\subset V$ are defined as the core region and fringe region of cluster $C_i$, respectively. These two sets divide the universe into three parts $Co\left(C_i\right),Fr\left(C_i\right)$ and $Tr\left(C_i\right)$, which chapter the three types of relationships between the objects and cluster, namely, objects belong to the cluster, objects not belong to the cluster, and objects partially belong to the cluster. For the samples in $Co\left(C_i\right)$, they belong to the cluster $\left(C_i\right)$ definitely and have a higher within-class similarities. For the samples in $Fr\left(C_i\right)$, they maybe belong to the cluster $\left(C_i\right)$ and have a lower similarities with the core samples. For the samples are in $Tr\left(C_i\right)$, they do not belong to the cluster $C_i$ definitely. Different from using one single set to represent the cluster, three-way clustering uses $Co\left(C_i\right),Fr\left(C_i\right)$ to represent one cluster. Three-way clustering addresses the problem of information uncertainty in traditional clustering methods through adding fringe region, which reduces decision risk caused by inaccurate information. We take Figure 2 as an example to show one three-way clustering result.

There are many different strategies to obtain the the core regions and fringe regions of three-way clustering. Typically, evaluation function is a commonly used method. The main idea of evaluation function is to construct a function and a pair of thresholds. The function assigns each sample a value. The samples with values greater than one threshold are assigned to the core region and the objects with values between the two thresholds are assigned to the fringe region. From a statistical point of view, a sample belongs to a certain cluster with a certain probability. It is reasonable to assign a sample to core region of one cluster when the probability of the sample belonging to this cluster is much greater than the probability of the sample belonging to other clusters. However, when the probability of a sample belonging to some clusters are almost same, it will be difficult to make a positive decision. Adopting a delayed decision and assigning them into the fringe region can reduce the risk of decision. Three-way clustering addresses the problem of information uncertainty in traditional clustering methods through adding fringe region, which reduces decision risk caused by inaccurate information. Recently, three-way clustering has attracted a lot of research, and many three-way clustering algorithms were developed. Wang et al. [37] proposed a three-way clustering framework by using contraction and expansion operators inspired in mathematical morphology; Jia et al. [38] introduced the definition of sample similarity to measure the uncertainty and developed an automatic three-way clustering approach. Fan et al. [39] proposed three-way density-sensitive spectral clustering algorithm by using density-sensitive metric. Shah [40] proposed a new three-way clustering by using image inspired cluster blur and sharp operators. Except the above research results, some other algorithms also enrich the theories and models of three-way clustering [41,42,43,44,45].

Given a set of data objects $U=\left\{x_1,x_2,\cdots ,x_n\right\}$, $C=\left\{c_1,c_2,\cdots ,c_k\right\}$ is a finite set of clusters, and U is divided into k classes. The idea of three-way clustering is to use a pair of sets to represent each cluster. This pair of sets consists of a core region (Co), a fringe region (Fr), and a trivial fringe (Tr) [46]. The results of three-way clustering can be represented as the following family of clusters:

$$\mathbb{C}=\{(Co\left(C_1\right),Fr\left(C_1\right)),(Co\left(C_2\right),Fr\left(C_2\right)),\cdots ,(Co\left(C_k\right),Fr\left(C_k\right))\}.$$

(3)

According to the definition of clustering results, $Co\left(C_i\right)$ and $Fr\left(C_i\right)$ must meet the following three conditions:

$$\begin{array}{c}\hfill \left(1\right)C\mathrm{o}\left(C_i\right)\ne \varphi ,i=1,2,\cdots ,k;\end{array}$$

(4)

$$\begin{array}{c}\hfill \left(2\right)\bigcup _{i=1}^k(Co\left(C_i\right)\cup Fr\left(C_i\right))=U;\end{array}$$

(5)

$$\begin{array}{c}\hfill \left(3\right)Co\left(C_i\right)\cap Co\left(C_j\right)=\varphi ,i\ne j.\end{array}$$

(6)

2.2. Three-Way k-Means

The traditional k-means algorithm is an iterative clustering analysis algorithm. The first step of the k-means algorithm is to randomly select k samples as the initial cluster centers. The second step is to calculate the distance between each sample and cluster center, and then assign the samples to the nearest cluster. The process is repeated by continuously updating the iterative cluster centers until a certain termination condition is met. In the process of k-means algorithm, there are only two relationships between samples and various clusters: belonging to the cluster and not belonging to the cluster. This two relationships ignore the samples which partially belong to one cluster. In fact, there is another type of relationship between sample and various clusters: belong to partially. In order to capture these three types of relationships, three-way k-means [21] was proposed by integrating three-way decision theory with the k-means algorithm. The main idea of the three-way k-mean algorithm is to introduce fault-tolerant errors in the k-means iteration process, and to represent the result of each cluster with a core region and a fringe region. The procedure of three-way k-means clustering consists mainly of two steps. The first step is to obtain the support of each cluster and the second step is to separate the core region from the support. For each object v and randomly selected k centroids $x_1,\cdots ,x_k$, let $d(v,x_j)$ be the distance between itself and the centroid $x_j$. Suppose $d(v,x_i)={min}_{1\le j\le k}d(v,x_j)$ and $T=\{j:d(v,x_j)-d(v,x_i)\le {\epsilon }_1$ and $i\ne j\}$, where ${\epsilon }_1$ is a given parameter. Then,

(1)

If $T\ne \varphi$, then $v\in \mathrm{support}\left(C_i\right)$ and $v\in \mathrm{support}\left(C_j\right)$.

(2)

If $T=\varphi$, then $v\in \mathrm{support}\left(C_i\right)$.

The modified centroid calculations for the above procedure are given by:

$$x_i=\frac{{\sum }_{v\in \mathrm{support}\left(C_i\right)}v}{\left|\mathrm{support}\right(C_i\left)\right|},$$

(7)

where $i=1,\cdots ,k$, v are all objects in $\mathrm{support}\left(C_i\right)$, and $\left|\mathrm{support}\right(C_i\left)\right|$ is the number of objects in $\mathrm{support}\left(C_i\right)$.

This process is repeated until modified centroids in the current iteration are identical to those that have been generated in the previous one, i.e., when the prototypes are stabilized. The second step is to separate the core regions from the supports using a perturbation analysis method. Algorithm 1 is designed to describe the process of TWKM clustering.

Algorithm 1: Three-way k-means [21]

3. The Improved Three-Way k-Means

The three-way k-means clustering algorithm integrates three-way decision theory with the k-means algorithm and uses a pair of sets to represent a cluster, which can effectively deal with the uncertainty of data. However, as for the k-means algorithm, the three-way k-means method is still sensitive to the initial clustering centers and can easily succumb to the problem of local optimization.To solve this problem, we present an improved three-way k-means clustering algorithm by combining a random probability selection strategy and the pheromone feedback mechanism in the ant colony algorithm with three-way k-means. The sensitivity of the three k-means clustering algorithm to the initial clustering center is optimized through continuous updating iterations, so as to avoid the clustering results easily falling into local optimization. The weights of the core domain and the boundary domain are dynamically adjusted to avoid the influence of artificially set parameters on the clustering results.

3.1. Random Probability Selection Strategy

The ant colony algorithm simulates the foraging behavior of ants in nature, and initially solves the problem of traveling salesman by analogy with a pheromone mechanism. The inspiration for the ant colony algorithm came from the behavior of ants in finding the shortest path in the process of looking for food. In the ant colony algorithm, the process of ants looking for food sources can be viewed as continuous clustering. According to the size of the current pheromone quantity, ants randomly select according to probability, and allocate samples to each cluster center. The larger the pheromone amount between a sample and a cluster center, the greater the probability that the sample will be assigned to this class. In the process of clustering, the probability that a sample is assigned to a cluster center is calculated according to the size of the pheromone between the sample and the cluster center and the heuristic function. The random probability selection strategy greatly increases the effectiveness of the algorithm, causes the algorithm to have the characteristics of convergence, and prevents the algorithm from falling into local optimization. The probability calculation formula of the cluster center to which the ant search sample belongs is:

$$p_{ij}={\tau }_{ij}^{\alpha }\left(t\right){\eta }_{ij}^{\beta }\left(t\right),$$

(8)

where ${\eta }_{ij}\left(t\right)$ is a heuristic function and ${\eta }_{ij}=\frac{1}{d_{ij}}(i=1,2,\dots n;j=1,2,\dots ,k)$, $d_{ij}$ represents sample $v_i$ to cluster center $x_j$. ${\tau }_{ij}\left(t\right)(i=1,2,\dots n;j=1,2,\dots ,k)$ represents the pheromone concentration between sample $v_i$ and cluster center $x_j$. The pheromone concentration is distributed between the sample and the clusters, and the initial pheromone concentration is 1, t is the number of iterations, $\alpha$ is the pheromone importance factor, and $\beta$ is the heuristic importance factor. To increase the diversity of search and to speed up the convergence speed, at the beginning, ants randomly select a sample $v_i$ as the starting point, and then Formula (7) is used to calculate the probability p of the sample to each cluster center $x_j$. The cluster of sample $v_i$ is determined using roulette. The above process is repeated for another sample until all the samples are traversed to form a solution.

In the ant colony algorithm, the objective function is used to evaluate the solution formed by all ants after completing an iteration, and only the clustering results obtained by the ants with the best objective function value are retained. We construct the fitness function using the intra-cluster cohesion function and the inter-cluster dispersion function, so that the objects in the same cluster are as similar as possible, and the objects in different clusters are as different as possible. The intra-cluster cohesion function is defined as follows,

$$J=\sum _{i=1}^k({\omega }_{icore}\sum _{x_i\in icore}(\parallel v_i-m_k{\parallel }^2)+({\omega }_{ifringe}\sum _{x_i\in ifringe}(\parallel v_i-m_k{\parallel }^2),$$

(9)

where J represents the sum of the distance between each sample $v_i$ and cluster center $x_j$, which is used to evaluate the degree of cohesion. ${\omega }_{icore}$ and ${\omega }_{ifringe}$ represent the weight values of the core region and the fringe region, respectively. Dynamic adjustment ${\omega }_{icore}$ and ${\omega }_{ifringe}$ can effectively avoid the influence of a sample’s number change in the core region and the fringe region, and can also avoid the influence of clustering centers due to differences in distance distribution. In this paper, we assume that ${\omega }_{icore}$ and ${\omega }_{ifringe}$ satisfy the following equations.

$${\omega }_{icore}+{\omega }_{ifringe}=1,$$

(10)

$$\frac{{\omega }_{icore}}{{\omega }_{ifringe}}=\frac{|{\omega }_{icore}|}{|{\omega }_{icore}|},$$

(11)

${\omega }_{icore}$ and ${\omega }_{ifringe}$ where $|{\omega }_{icore}|$ and $|{\omega }_{icore}|$ are the number of samples in the core region and the fringe region, respectively.

The quality of clustering results is determined by the intra-cluster distance and the inter-cluster distance; when the intra-class distance is smaller, the inter-class distance is larger, the value of the objective function is smaller, and the clustering result is better. The inter-cluster dispersion function is defined as,  

$$D=\frac{1}{k}(\sum _{i,j=1}^k\parallel x_i-x_j{\parallel }^2),$$

(12)

where $x_i$ and $x_j$ represent the cluster center of $C_i$ and $C_j$, respectively.

Based on the intra-cluster cohesion function and the inter-cluster dispersion function defined by (8) and (10), we construct the following fitness function to optimize three-way k-means:

$$Fit=\frac{1+D}{1+J}.$$

(13)

In the process of looking for food sources, ants release pheromones on the paths they pass. The higher the pheromone concentration, the shorter the distance of the road. In this way, the more ants walk, the higher the pheromone concentration on this path. Each ant moves towards the direction with the highest pheromone concentration, and the continuously strengthened pheromone attracts more ants, so a positive feedback mechanism is formed. As time goes on, the pheromone on the poor path cannot be strengthened, and as the pheromone volatilizes continuously, it loses its attraction, thus forming a negative feedback mechanism.

The positive feedback mechanism attracts more ants to choose the current optimal path, accumulates more pheromone, increases the probability of other ants choosing the path, narrows the scope of ant search, and promotes the convergence of the clustering algorithm. The negative feedback mechanism can eliminate the effect of the positive feedback mechanism, effectively preventing more ants from being attracted to the optimal path, and making the algorithm result fall into the local optimal solution. Using the positive and negative feedback of pheromones, the ant colony algorithm avoids the algorithm falling into a local optimal solution and increases the diversity of solutions.

Pheromone updating in the ant colony algorithm uses the overall information of the ant colony. When the ant releases the pheromone, the pheromone remaining on the path will gradually disappear. This is also to make the next generation of ants more robust both globally and locally when choosing the path. Therefore, when all the ants have completed a cycle, the global update of the residual pheromone is carried out. The pheromone update formula in the ant colony algorithm is as follows:

$${\tau }_{ij}(t+1)=(1-\rho )\ast {\tau }_{ij}\left(t\right)+\Delta {\tau }_{ij},$$

(14)

$$\Delta {\tau }_{ij}=0.1\ast {\tau }_{ij},$$

(15)

where, parameter $0<\rho <1$ indicates the degree of volatilization of the pheromone, ${\tau }_{ij}$ represents the pheromone concentration, and $\Delta {\tau }_{ij}$ represents the increment in pheromone.

3.2. The Improved Three-Way k-Means Algorithm

Because the clustering results of the standard three-way k-means algorithm depend on the selection of initial centers they easily succumb to the problem of local optimization. To overcome this problem, this paper presents an improved three-way k-means algorithm by integrating the ant colony algorithm and three-way k-means. An original element of this paper is the application of clustering centers obtained by the ant colony algorithm to the three-way k-means, which makes up for the shortcomings of the three-way k-means clustering algorithm due to the random selection of clustering centers. Figure 3 presents a flowchart of the proposed algorithm.

The ant colony algorithm simulates the foraging behavior of ants in nature, and initially solves the problem of traveling salesman by analogy with a pheromone mechanism. Ants randomly select a sample in the sample space as the starting point. The probability that a sample is assigned to a cluster is obtained according to the amount of the pheromone between the sample and the cluster center. The sample is allocated to a cluster by roulette. Then the ant selects another sample until all samples are assigned, that is, when an iteration is completed to form a solution. The optimal solutions are calculated using the value of the objective function. The pheromone in the ant colony algorithm reflects the overall information in the ant colony. When the ant releases the pheromone, the pheromone remaining on the path will gradually disappear. This is also to make the next generation of ants more robust globally and locally when choosing the path. The specific steps of the algorithm are shown in Algorithm 2:

The detailed complexity of Algorithm 2 is as following: Line 3 to Line 11 are to find the support of each cluster. The time complexity of this process is $O\left(knm\right)$, where n and m are the number of elements and attributes, respectively. Line 12 is to separate the core regions from the support sets using centroid perturbation analysis. The time complexity of this process is $O\left(knm\right)$. Line 13 to Line 14 are to update the process. The time complexity of Algorithm 2 is $O\left(tknm\right)+O\left(knm\right)$, where t is the number of iterations.

Algorithm 2: The improved three-way k-means based on ant colony algorithm.

4. Experimental Analysis

4.1. Evaluation Indices

• Accuracy ($Acc$) [47]

  $$Acc=\frac{1}{n}\sum _{i=1}^k{C}_i$$

  (16)
  In the above formula, n is the total number of samples in the dataset, $C_i$ is the correct number of samples divided into class clusters i, and k is the number of class clusters. $Acc$ represents the ratio between the number of correctly partitioned elements and the total number. A greater $ACC$ value implies a better clustering result. When $ACC$ = 1, the result of the clustering algorithm is consistent with the real result.
• Davies–Bouldin index ($DBI$) [47].

  $$DB=\frac{1}{c}\sum _{i=1}^c\underset{j\ne i}{max}\left\{\frac{S\left(C_i\right)+S\left(C_j\right)}{d(x_i,x_j)}\right\}$$

  (17)

  where $S\left(C_i\right)$ and $d(x_i,x_j)$ are the intra-cluster distance and the inter-cluster separation, respectively. $S\left(C_i\right)$ is defined as follows:

  $$S\left(C_i\right)=\frac{{\sum }_{v\in C_i}\Vert v-x_i\Vert }{\mid C_i\mid }.$$

  (18)
  As a function of the ratio of the within cluster scatter to the between cluster separation, a lower value will mean that the clustering is better.
• Average silhouette index ($AS$) [47].

  $$AS=\frac{1}{n}\sum _{i=1}^n{S}_i,$$

  (19)

  where n is the total number of objects in the set and $S_i$ is the silhouette of object $v_i$, which is defined as,

  $$S_i=\frac{b_i-a_i}{max\{a_i,b_i\}},$$

  (20)
  $a_i$ is the average distance between $x_i$ and all other objects in its own cluster, and $b_i$ is the minimum of the average distance between $x_i$ and objects in other clusters. The range of the average silhouette index is $[-1,1]$; a larger value means a better clustering result.

4.2. Performances of Proposed Algorithm

To test the performances of our proposed algorithm, we employed 10 datasets from the UCI Machine Learning repository, which were Wine, Class, Ecoli_Nor, Forest, Bank, Iris, Contraceptive, Molecular Biology1, Libras_Nor, and Caffeine_consumption.

Table 1. A description of dataset used.

ID	Datasets	Samples	Attributes	Classes
1	Wine	178	13	3
2	Class	214	9	6
3	Ecoli	366	7	8
4	Forest	523	27	4
5	Bank	1372	4	2
6	Iris	150	4	3
7	Contraceptive	1473	9	3
8	Molecular Biology	106	52	2
9	Libras	360	90	15
10	Caffeine Consumption	1885	12	7

shows the details of these datasets.

Because the evaluation indices $Acc$, $DBI$ and $AS$ are only adopted for the hard clustering results, three-way clustering cannot calculate these values directly. To present the performances of our proposed algorithm, we use the core regions to form a clustering result, then calculate the $Acc$, $DBI$ and $AS$ values using the core region to represent the corresponding cluster. The average $ACC$, $DBI$ and $AS$ values are achieved by running the process 30 times on all datasets. To compare clustering effects, the performances of k-means [19], FCM [48], and three-way k-means [21] are also presented in

Table 2. The performances of average $Acc$ value.

ID	Data Sets	k-Means	FCM	Three-Way k-Means	Proposed Algorithm
1	Wine	0.6573	0.6692	0.6831	0.6911
2	Class	0.5981	0.6007	0.6112	0.6366
3	Ecoli	0.6339	0.6335	0.6652	0.6773
4	Forest	0.7795	0.7540	0.7807	0.8294
5	Bank	0.5758	0.5969	0.6123	0.6131
6	Iris	0.8866	0.8933	0.9040	0.9040
7	Contraceptive	0.2145	0.2179	0.2822	0.2826
8	Molecular Biology	0.6037	0.6226	0.6547	0.6659
9	Libras	0.8611	0.9162	0.9256	0.9240
10	Caffeine Consumption	0.2005	0.1960	0.2411	0.2422

,

Table 3. The performances of average $DBI$ value.

ID	Data Sets	k-Means	FCM	Three-Way k-Means	Proposed Algorithm
1	Wine	1.7835	1.6922	1.5542	1.5431
2	Class	1.0475	1.2233	0.7855	0.7596
3	Ecoli	1.1504	1.0273	0.9667	0.9425
4	Forest	1.2774	1.2253	1.200	1.1879
5	Bank	1.1913	1.1952	1.1332	1.1267
6	Iris	0.7609	0.7507	0.7236	0.7355
7	Contraceptive	1.2716	1.2539	1.2323	1.2220
8	Molecular Biology	4.9588	4.8236	4.6783	4.6689
9	Libras	1.9240	1.9126	1.9033	1.9023
10	Caffeine Consumption	1.9116	1.8072	1.6655	1.6048

and

Table 4. The performances of average $AS$ value.

ID	Data Sets	k-Means	FCM	Three-Way k-Means	Proposed Algorithm
1	Wine	0.3383	0.2337	0.3347	0.3574
2	Class	0.5309	0.5543	0.5887	0.6038
3	Ecoli	0.4419	0.4326	0.4433	0.4524
4	Forest	0.4029	0.4302	0.4559	0.4669
5	Bank	0.5000	0.4954	0.5111	0.5280
6	Iris	0.6959	0.7091	0.7114	0.7188
7	Contraceptive	0.4236	0.4309	0.4597	0.4672
8	Molecular Biology	0.0553	0.0538	0.0558	0.0585
9	Libras	0.3519	0.3000	0.3533	0.3556
10	Caffeine Consumption	0.3150	0.3491	0.3517	0.3563

. The best performance for each dataset is highlighted in bold.

To analyze the time comparison of different algorithms, Figure 4 lists the running time of different algorithms on the 10 UCI datasets where the unit of measurement for time is “second”.

4.3. Experimental Results Analysis

The average $Acc$ values of k-means, FCM, three-way k-means and the proposed algorithm are reported in Table 2. Obviously, comparing with other three algorithms, the proposed algorithm achieves a better $ACC$ value on 8 data sets. Though the performances of $Acc$ value on Iris and Libras by three-way k-means algorithm are equal to or superior to the results by the proposed algorithm, while the $DBI$ and $AS$ value of our proposed algorithm are better than the results of three-way k-means. The increase of $Acc$ value indicates that the proposed algorithm assign more samples to right cluster than other algorithms.

The results of $DBI$ and $AS$ are listed in Table 3 and Table 4, respectively. From Table 3 and Table 4, we can find that the proposed algorithm obtains better results than other algorithms in terms of $DBI$ and $AS$ value on all data sets. Since $DBI$ is the ratio of the within cluster scatter to the between cluster separation, the improvement of $DBI$ means that the clustering results of proposed algorithm have a higher cluster separation. The results of $AS$ can also verify the fact that the clustering results of proposed algorithm have a lower intra class distance and a higher inter class distance. This can be attributed to the fact that the clustering result of the proposed algorithm is represented by core regions when calculate the $DBI$ value and $AS$ value, which can increase the degree of separation between clusters and reduce the degree of dispersion within clusters.

The running time of the proposed algorithm is much longer than k-means and FCM on all the UCI datasets, as shown in Figure 4. This is because the time complexity of the proposed algorithm is much more than that of k-means and FCM. Compared with the three-way k-means method, the proposed algorithm takes slightly longer. This is because the proposed algorithm integrates the ant colony algorithm into three-way k-means, which adds to the running time of the proposed algorithm.

The above discussion suggests that the proposed algorithm can effectively improve the results of three-way k-means, solving the clustering problem of uncertain elements and maintaining improved clustering performance at the same time. Though the proposed algorithm can achieve better clustering results than three-way k-means, it still has the following two disadvantages:

• Similar to the k-means algorithm, the proposed method can achieve good results for convex datasets. If the dataset is non-convex, the proposed algorithm fails to give good results.
• The time complexity and computation complexity of the proposed algorithm are higher than for k-means and three-way k-means, which means it is not suitable for big data.

5. Conclusions and Future Work

Three-way clustering uses a core region and a fringe region to represent a cluster, which divide the universe into three disjoint sets to reflect the three types of relationship between an object and a cluster. The samples in the core region are determined to belong to this type of cluster, while the samples in the fringe region may belong to this type of cluster. Three-way clustering assigns the samples with uncertainty information into corresponding fringe regions, which reduces the risk of decision-making. However, the standard three-way k-means algorithm does not always guarantee good results, as the accuracy of clustering depends on the selection of the initial centroids and easily succumbs to the problem of local optimization. Aiming to solving this problem, this paper presents an improved three-way k-means algorithm by integrating the ant colony algorithm and three-way k-means. The idea of this algorithm is to apply the clustering center obtained by each iteration of the ant colony algorithm to the three-way k-means, which compensates for the shortcomings of the three-way k-means clustering algorithm due to the random selection of clustering centers. The proposed algorithm optimizes three-way k-means using a random probability selection strategy in the ant colony algorithm and the positive and negative feedback mechanism of pheromones to dynamically adjust the weight. The experiments on UCI datasets show that the proposed algorithm can improve the performance of the three-way k-means clustering results according to the comparison of $ACC,DBI$ and $AS$.

Finally, we should point out that the standard three-way k-means algorithm has two problems, the selection of the initial clustering centers and the determination of the cluster number. This paper presents a method to improve the selection of the initial clustering centers. The determination of cluster number is an interesting topic that requires further investigation.