Visual Assessment of Cluster Tendency with Variations of Distance Measures

Abstract

Finding the cluster structure is essential for analyzing self-organized networking structures, such as social networks. In such problems, a wide variety of distance measures can be used. Common clustering methods often require the number of clusters to be explicitly indicated before starting the process of clustering. A preliminary step to clustering is deciding, firstly, whether the data contain any clusters and, secondly, how many clusters the dataset contains. To highlight the internal structure of data, several methods for visual assessment of clustering tendency (VAT family of methods) have been developed. The vast majority of these methods use the Euclidean distance or cosine similarity measure. In our study, we modified the VAT and iVAT algorithms for visual assessment of the clustering tendency with a wide variety of distance measures. We compared the results of our algorithms obtained from both samples from repositories and data from applied problems.

1. Introduction

The identification of the cluster structure in data is useful for discovering the dependencies between the sample objects. Dividing the sample into groups of similar objects simplifies further data processing and decision making, and enables us to apply a separate analysis method to each cluster. A decrease in the number of groups can help us to select the most common patterns in the data. Finding the cluster structure is essential for analyzing self-organized networking structures, such as social networks. In such problems, different measures of distance may be applied.

For evaluating the similarity between objects, we can use the internal and external similarity measures [1]. Measures of internal similarity are based on the proximity of objects within a group and their distance from objects in other groups. Internal similarity measures rely on information obtained only from a dataset. External similarity measures, on the other hand, are based on comparing clustering results with reference results, usually created with the assistance of experts. In this paper, we focus on internal similarity measures. The modern literature provides a wide range of evaluation criteria, such as the Calinski–Harabasz index [2], Dunn index, Davies–Bouldin index [3], silhouette index [4,5], and others.

In the work of Bezdek and Hathaway [6], the authors raised an important question: “Do clusters exist?” Clustering algorithms divide data points into groups according to some criterion so that objects from one group are more similar to each other than to objects from other groups. Clustering algorithms are usually applied in an unsupervised way when vectors of object parameter values or a matrix of differences between objects are used as initial data.

In unsupervised learning, clustering methods form groups of objects, even if the analyzed dataset is a completely random structure. This is one of the most important problems of clustering. Therefore, the first validation task that is recommended to be performed before clustering is to assess the general propensity of the data to cluster (clustering tendency). Before evaluating the performance of clustering, we must make sure that the dataset we are working with tends to cluster and does not contain uniformly distributed points (objects). If the data do not tend to cluster, then the groups identified by any modern clustering algorithm may be meaningless.

The term “clustering tendency” is introduced in [7]. Using it, we can estimate the propensity of the data points to clustering.

Some methods for assessing clustering propensity are discussed in [7,8]. They can be roughly divided into following categories: statistical and visual. In [9], the authors noted that “A clustering index should be an indicator of the degree of non-uniformity of the distribution of the objects. It should be used both as an automatic warning or, better, as a quantitative measure of the quality of the dataset, original or extracted”. The Hopkins clustering index [10,11] has these properties [9]. It is based on the null hypothesis H₀: “The points in a dataset are uniformly distributed in multidimensional space”. In a known dataset, X₁, …, X_N, in a M-dimensional space, B real points are randomly selected and artificial points are added using a distribution with the standard deviation of known data points. For each real and artificial point, the Euclidean distance to the nearest point is determined. The distance between a real point to its nearest point $d_b$ is compared with the distance between an artificial point to nearest real point $D_a$. If there is a clustering tendency, then $d_b<D_a$ [12]. The found minimal distances are summarized for all points and used for the Hopkins clustering index, which is calculated as follows:

$$H_{ind}=\frac{{{\displaystyle \sum }}_{a=1}^B{D}_a}{{{\displaystyle \sum }}_{b=1}^B{d}_b+{{\displaystyle \sum }}_{a=1}^B{D}_a},$$

where $d_b=\sqrt{{\left(X_b-X_k\right)}^2}$ is the Euclidean distance between each of these real points b and their nearest real neighbor k; $D_a=\sqrt{{\left(X_a-X_k\right)}^2}$ is the Euclidean distance between these artificial points a and the nearest real point k, $H_{ind}\in \left[0,1\right]$. If $H_{ind}=0.5$, then the data points are scattered randomly and it is impossible to extract the cluster structure. If $H_{ind}=0$, then the data are homogeneous and there are no clusters, so we can say that all the data make up one big cluster. Approximation of $H_{ind}$ to 1 means that a cluster structure can be identified in the data with a high degree of probability; if $H_{ind}=1$, then this is the maximum degree of clustering. To obtain a stable Hopkins clustering index value (mean value), multiple calculations are carried out with new random samples.

In [9,12], the authors presented the modified Hopkins clustering index. This index is calculated as follows:

$$H_{ind}^{*}=\underset{B\to \infty }{\lim }\frac{\left({{\displaystyle \sum }}_{a=1}^B\raisebox{1ex}{$D_a$}\!\left/ \!\raisebox{-1ex}{$B$}\right.\right)-\left({{\displaystyle \sum }}_{n=1}^{N}d\raisebox{1ex}{$d_n$}\!\left/ \!\raisebox{-1ex}{$N$}\right.\right)}{\left({{\displaystyle \sum }}_{a=1}^B\raisebox{1ex}{$D_a$}\!\left/ \!\raisebox{-1ex}{$B$}\right.\right)+\left({{\displaystyle \sum }}_{n=1}^{N}d\raisebox{1ex}{$d_n$}\!\left/ \!\raisebox{-1ex}{$N$}\right.\right)},$$

where $d_n=\sqrt{{\left(X_n-X_k\right)}^2}$ is the Euclidean distance between these points n and their real nearest point k, i.e., $d_n$ is computed for all the N points,$H_{ind}^{*}\in \left[0,1\right]$. If $H_{ind}^{*}=0$, then it is impossible to distinguish a cluster structure in the data points, if $H_{ind}^{*}=1$, then this is the maximum degree of clustering [9].

In [9], the minimum spanning tree (MST) clustering index is proposed. The clustering algorithms associated with MST determine whether a point belongs to a cluster based on connectivity. The Prim’s algorithm [13] and the Kruskal algorithm [14] are used to calculate the MST. The MST clustering index is defined as follows [9]:

$$Inde{x}_{MST}={\displaystyle \sum }_{d>d_{crit}}\frac{d}{d_{crit}}-1,$$

where d_crit is the critical distance value and d is the tree distance. If $Inde{x}_{MST}=0,$ then we accept the null hypothesis H₀, otherwise, the dataset is not homogeneous. The MST clustering index can be used to find groups, to measure the degree of heterogeneity of data points, and to detect the outliers.

The authors [9] noted that the MST clustering index is sensitive to the empty space outside the multivariate limits of data points. In the case when there are areas of points in the dataset that do not have clear boundaries, the $Inde{x}_{MST}$ and $H_{ind}^{*}$ indexes are unable to determine the clustering tendency. However, all of these methods estimate only tendency to clustering, but not the number of clusters.

For fuzzy clustering approach, there are several methods for estimating cluster validity. For example, Xie-Beni’s separation index (XBI) [15], which expresses the inverse ratio between the total variation and N times the minimum separation of the clusters:

$$XBI=\frac{1}{\left(\frac{{{\displaystyle \sum }}_{k=1}^K{{\displaystyle \sum }}_{i=1}^N{\mu }_{ik}\Vert x_i-v_k\Vert {}^2}{N·d_{min}^2}\right)},$$

where d_min is the minimum distance of all distances between two clusters, K is the number of clusters, and ${\mu }_{ik}$ is calculated as follows:

$${\mu }_{ik}=\{\begin{array}{c}1, if \Vert x_i-u_k\Vert \le \Vert x_i-u_j\Vert ,\\ 0, otherwise\end{array}1\le k,j\le K, j\ne k, 1\le i\le N.$$

In [6], the authors presented a tool, named VAT (visual assessment of tendency), for visual assessment of the clustering tendency. At the first step, VAT algorithm reorders the matrix of differences. At the second step, it generates a reordered dissimilarity image (RDI)—a cluster heat map. In a cluster heat map, dark areas located on the main diagonal enable us to determine the possible number of groups [16]. The VAT can be used for any numerical datasets. The VAT algorithm is similar to Prim’s algorithm [13] for determining an MST of a weighted graph. The authors of the VAT algorithm noted the two following differences. Firstly, the VAT algorithm does not require their MST representation, only the order in which vertices are added as the graph grows is important. Secondly, in the VAT algorithm, the choice of the initial vertex depends on the maximum weight of the edge. In cases of a complex data structure, the efficiency of the VAT algorithm decreases quickly. However, VAT works well for datasets with dimension N ≤ 500. For datasets with higher dimensions, the time complexity of the algorithm (O(N²)) increases.

In [17], the authors proposed an improved VAT (iVAT) algorithm. The iVAT images more clearly showed the number of groups, as well as their sizes. As the authors assure in [17]: “Based on the iVAT image, the cluster structure in the data can be reliably estimated by visual inspection”. In addition, the authors proposed automated VAT (aVAT). The article [18] describes a method that handles asymmetric dissimilarity data (asymmetric iVAT (asiVAT)). In [19], the authors presented an approach for finding the number of groups by traversing the MST backwards by cutting (k – 1) the largest edge in the MST.

VAT and iVAT have two restrictions:

1)

RDI may not be representative if the cluster structure in the data points is complex.

2)

The quality of the RDI in VAT is significantly degraded due to the presence of outliers.

The revised visual assessment of cluster tendency (reVAT) algorithm [20], using a quasi-ordering of the N points, reduces the time complexity to O(n). The reVAT does not require reordering the matrix of differences. The reVAT plots a set of c profile graphs of specific rows of an ordered difference matrix. An ensemble of a set of c profile graphs is used to visually determine the number of groups [20]. However, when the number of clusters is large and there are areas of points that do not have clear boundaries, a visual assessment of the number of clusters is difficult with the use of reVAT.

Research [21] has presented a visual assessment of the clustering trend for large and relational datasets (bigVAT). This method uses the quasi-ordering from reVAT and builds an image similar to that built by VAT. The bigVAT solves the problems of the VAT and reVAT methods [20]. However, the image it builds may not be as visual as the image ordered by VAT.

In [22], the authors presented the scalable VAT (sVAT) algorithm. To create an image, the sVAT algorithm generates a representative sample containing a structure of clusters similar to that of the original dataset. For this sample, sVAT generates a VAT-ordered image considering the following parameters: an overpriced grade of the actual number of clusters (k) and the size of representative sample (n) from the full set of N points. The sample is formed as follows: a set of k distinguished points is selected, then additional data are added next to each of the k objects. In addition, [23] described the method for visual assessment of the clustering tendency for rectangular matrices of differences (coVAT).

The cluster count extraction (CCE) algorithm [24] creates a visual image using the VAT algorithm. This algorithm further processes the image to improve it for automatic verification, and extracts the number of clusters from the improved image. The article [25] demonstrates the dark block extraction (DBE) method. This method generates an RDI (VAT image) and then segments the desired areas into RDI and converts the filtered image to a distance transform image. The transformed image is superimposed on the RDI diagonal and the potential number of groups is determined. The CCE and DBE algorithms automatically determine the number of groups in unlabeled data points.

Paper [26] presents the CLODD (cluster in ordered difference data) method. In this algorithm, to identify potential clusters, an objective function is determined that combines contrast and edginess measures and is optimized using particle swarm optimization (PSO) [27]. With the help of the objective function, the block structure is recognized in the reordered dissimilarity matrix. The CLODD algorithm is used both to search for clusters in unlabeled data and to represent the validity index of these clusters.

The effectiveness of the considered methods based on the SDS largely depends on the quality of the SDS images. SDS is effective only for defining compact clearly marked groups. However, there are datasets that have a very complex structure of clusters.

In [28], the authors presented a novel silhouette-based clustering propensity score (SACT) algorithm for determining the potential number of clusters and their centroids in applications for hyperspectral image analysis. The SACT algorithm was inspired by the VAT algorithm. The SACT algorithm builds a weighted matrix and a weighted graph using the Euclidean distance measure, then, a minimum spanning tree is constructed using Prim’s algorithm. The original minimum spanning tree is then hashed into branches corresponding to data clusters. Finally, the number of potential clusters is determined using the silhouette index.

Clustering datasets with different cluster density is a difficult problem. The VAT and iVAT algorithms use the difference between intra-cluster and inter-cluster distances to discover the data structure. These algorithms do not work well for data points consisting of groups with different levels of density. In [29], the authors introduced the locally scaled VAT (LS-VAT) algorithm and locally scaled iVAT (LS-iVAT) algorithm. In the LS-VAT algorithm, before generating the MST, the distance matrix is converted into an adjacency matrix using local scaling [30]. The algorithm builds a similarity graph for a dataset; for this, instead of connecting the data point that has the smallest distance from the current tree, it connects the data point that has the highest adjacency to the existing tree. In case a set of points with different inter-cluster densities is used, the LS-iVAT algorithm must be used to provide higher quality iVAT images. LS-VAT and LS-iVAT outperform algorithms [6,16,31,32] in terms of clustering quality.

In [33], the authors presented a semi-supervised constraint-based approach for the iVAT (coniVAT) algorithm. To improve VAT/iVAT for complex data points, coniVAT uses partial background knowledge in the form of constraints. ConiVAT uses the input constraints to learn a basic similarity measure, builds a minimum transitive matrix of differences, and then applies VAT to it.

For big datasets, algorithms ClusiVAT [34] and S-MVCS-VAT [35] were developed. In [36], the spectral technique was applied to ClusiVAT and S-MVCS-VAT algorithms. An algorithm based on VAT for streaming data processing was presented in [37].

Kumar and Bezdek [15] presented a detailed and systematic review of a variety of VAT and iVAT algorithms and models. This article describes 25 algorithms.

Thus, VAT/iVAT algorithms are methods for visually extracting some information about the structure of clusters from the initial dataset before applying any clustering algorithm. They do not change the initial dataset, but rearrange the objects in such a way as to emphasize the possible structure of the cluster.

Most of the considered algorithms use the Euclidean distance measure. However, it is known that, depending on the shape of the clusters, different distance measures are better suited for different datasets. Despite the fact that there are many distance measures (similarity measures) in the literature [16,18,38], it is quite difficult to choose the right one for a specific dataset. The choice of the distance measure that is most appropriate in each specific case allows obtaining clearer RDIs and, accordingly, obtaining a more valid result of pre-clustering. In our work, we propose a modification of VAT and iVAT algorithms convenient for using other distance measures than Euclidean. We discover the behavior of modified algorithms on the applied dataset and datasets from the repositories.

Organization of the paper is as follows. In Section 2, we give a description of the distance measures used and algorithms VAT and iVAT. In Section 3, we present the results of computational experiments both on the repository and applied data. In Section 4, we conclude the essence of the work.

2. Materials and Methods

Consider a set of points where each point is represented as a characteristic vector of dimension p, x_i ∈ R^p, X = {x₁,.., x_n} ∈ R^p. The second way to represent the points is the dissimilarity matrix D = [d_ij] (1), of n × n dimension, where d_ij is difference between the points I and j, calculated using a distance measure.

The dissimilarity matrix has several properties:

• Symmetry about the diagonal. The dissimilarity matrix is a square symmetrical n × n matrix with d_ij element equal to the value of a chosen measure of distinction between the ith and jth objects d_ij = d_ji.
• The distance values in the matrix are always non-negative d_ij ≥ 0.
• Identity of indiscernibles. In the matrix, the distinction between an object and itself is set to zero (d_ii is diagonal element, where d_ii = 0).
• The triangle inequality takes the form d_ij + d_jk ≥ d_ik ∀ i,j,k.

$$D=\left(\begin{array}{ccc}\begin{array}{c}0\\ ⋮\\ \begin{array}{c}{d}_{i1}\\ ⋮\\ d_{n1}\end{array}\end{array}& \begin{array}{c}\cdots \\ \cdots \\ \begin{array}{c}\cdots \\ \cdots \\ \cdots \end{array}\end{array}& \begin{array}{ccc}\begin{array}{c}{d}_{1j}\\ ⋮\\ \begin{array}{c}0\\ ⋮\\ d_{nj}\end{array}\end{array}& \begin{array}{c}\cdots \\ \cdots \\ \begin{array}{c}\cdots \\ \cdots \\ \cdots \end{array}\end{array}& \begin{array}{c}{d}_{1n}\\ ⋮\\ \begin{array}{c}{d}_{in}\\ ⋮\\ 0\end{array}\end{array}\end{array}\end{array}\right),$$

(1)

where d_ij is the difference between the ith and jth points.

The quality of the cluster solution also depends on the chosen distance measure. In this work, we used some distance measures (the Euclidean distance, the squared Euclidean distance, the Manhattan distance, the Chebyshev distance) based on the Minkowski function (l_p-norm) [39,40,41]:

$$d(x,y)={\left({\displaystyle \sum _{i=1}^M{\left|x_i-y_i\right|}^p}\right)}^{\frac{1}{p}},$$

(2)

where x and y are input vectors of dimension M. For parameter p, the following statement is true (proof): for p ≥ 1 and p = ∞, the distance is a metric; for p < 1, the distance is not a metric.

For p = 2, the function takes the form of Euclidean distance (l₂ norm):

$$d(x,y)=\sqrt{{\displaystyle \sum _{i=1}^M{\left(x_i-y_i\right)}^2}}.$$

(3)

The Euclidean distance is the most understandable and interpretable measure of the difference between points represented by feature vectors in multidimensional space. Therefore, the Euclidean distance and squared Euclidean distance are widely used for data analysis. The squared Euclidean distance is defined as:

$$d(x,y)={\displaystyle \sum _{i=1}^M{\left(x_i-y_i\right)}^2}.$$

(4)

We also used the standardized Euclidean distance, calculated as follows:

$$d(x,y)=\sqrt{{\displaystyle \sum _{i=1}^M{\left(\frac{x_i-y_i}{s_i}\right)}^2}}$$

(5)

where s_i is standard deviation of the ith characteristic of the input data vectors.

For p = 1, we obtained the Manhattan distance (l₁ -norm), which is the second most popular distance:

$$d(x,y)={\displaystyle \sum _{i=1}^M\left|x_i-y_i\right|}.$$

(6)

For $p=\infty$, the function calculates the Chebyshev distance [42], also called the chessboard distance:

$$d\left(x,y\right)=max\left|x_i-y_i\right|.$$

(7)

Most modern papers are devoted to problems that use the Euclidean or Manhattan distances.

There are other ways to calculate distances that do not depend on the parameter p and are not determined by the Minkowski function. For example, we used the Mahalanobis distance [43], the correlation-based distance (the correlation distance) [44], the cosine similarity [45], the Bray–Curtis dissimilarity [46], the Canberra distance [42].

Mahalanobis distance is defined as

$$d\left(x,y\right)=\sqrt{{{\displaystyle \sum }}_{i=1}^M{\left(x_i-y_i\right)}^T{C}^{-1}\left(x_i-y_i\right)},$$

(8)

where C is the covariance matrix calculated as

$$C=\mathrm{cov}(x,y)=\mu \left[\left(x-\mu (x)\right)\left(y-\mu (y)\right)\right],$$

(9)

where μ is the mean value.

The correlation distance is defined by:

$$d\left(x,y\right)=1-\frac{\left(x-\overline{x}\right)·\left(y-\overline{y}\right)}{{\Vert \left(x-\overline{x}\right)\Vert }_2{\left(\Vert y-\overline{y}\right)\Vert }_2},$$

(10)

where $\overline{x}$ and $\overline{y}$ are the means of the elements of x and y, respectively, and ${\Vert \left(x-\overline{x}\right)\Vert }_2, {\Vert \left(y-\overline{y}\right)\Vert }_2$ means norm, which is, by default, a second-order (Euclidean) norm.

The measure of similarity can be estimated through the cosine of the angle between two vectors:

$$d\left(x,y\right)=1-\frac{x·y}{{\Vert x\Vert }_2{\Vert y\Vert }_2},$$

(11)

where ${\Vert x\Vert }_2, {\Vert y\Vert }_2$ means norm, which is, by default, a second order (Euclidean) norm.

Bray–Curtis dissimilarity [46] is defined by:

$$d\left(x,y\right)=\frac{{{\displaystyle \sum }}_{i=1}^M\left|x_i-y_i\right|}{{{\displaystyle \sum }}_{i=1}^M\left|x_i+y_i\right|}.$$

(12)

Canberra distance [42] is given as follows:

$$d\left(x,y\right)={{\displaystyle \sum }}_{i=1}^M\frac{\left|x_i-y_i\right|}{\left|x_i\right|-\left|y_i\right|}.$$

(13)

VAT [6,16] and iVAT algorithms [47] are tools for visual assessment of the tendency to clustering of a set of points by reordering the dissimilarity matrix such that possible clusters appear as dark boxes diagonally in the cluster heat map. This cluster heat map can be used to visually estimate the number of clusters in a dataset [33]. VAT reordering is related with clusters created using single linker hierarchical clustering (SL), which makes it possible to extract SL-aligned groups from images of VAT (iVAT) [48].

The VAT algorithm is presented below (Algorithm 1) [6,15].

Algorithm 1. Visual assessment of the clustering tendency (VAT).

Required: A square n × n dissimilarity matrix D = [dij] satisfying: dij ≥ 0, dij = dji ∀ i,j, dii = 0 ∀ i. Set K = {1, …, n}, I = J = ∅ Select $\left(i,j\right)\in arg\underset{k\in K, q\in K}{\max }{d}_{kq}$ Set Pi = i, I = {i}, J = K − {i} for r = 2 to n do Select $\left(i,j\right)\in arg\underset{k\in I, q\in J}{\min }{d}_{kq}$ Set Pt = j, Replace I = I ⋃ {j}, J = J − {j}, dt−1 = dij endfor for p = 1 to n do for q = 1 to n do d*p,q = dPp,Pq endfor endfor Generate VAT image of D*: I (D*) Output: A square n × n reordered dissimilarity matrix $D^{*}=\left[d_{ij}^{*}\right]$, I (A*), P is reordered indices of matrix A.

In [16], the authors proposed an improved VAT (iVAT). This algorithm transforms the dissimilarity matrix with the idea, which is the following: when two distant objects are connected by a chain of other objects closely located, to reflect this connection, the distance d_ij between them has to be reduced. According to this correction, two objects, if connected by a set of successive objects forming dense regions, should be considered coming from one cluster [15]. The iVAT algorithm is presented in Algorithm 2 [15,16].

Algorithm 2. Improved visual assessment of the clustering tendency (iVAT).

Required: A square n × n reordered dissimilarity matrix $D^{*}=\left[d_{ij}^{*}\right]$ for r = 2 to n do $j=arg\underset{1\le k\le r-1}{\min }{d}^{*}{}_{rk}$ $A_{rj}^{\prime *}=A_{rj}^{*}$ $c=\left\{1,\dots ,r-1\right\}-\left\{j\right\}$ $d_{rc}^{\prime *}=\max \{d_{rj}^{*},d_{jc}^{\prime *}\}$ endfor $d_{rc}^{\prime *}=d_{cr}^{\prime *}$

In this research, we modified the VAT and iVAT algorithms in such a way to be able to calculate many possible distance measures besides the Euclidean (DVAT or DiVAT, see Algorithm 3).

Algorithm 3. DVAT (DiVAT) algorithm.

Required: Set of initial data vectors X1, …, Xn, where n is the number of points. Select distance measure. Calculate dissimilarity matrix D = [dij]. Run VAT (iVAT) algorithms.  Output: A square n × n reordered dissimilarity matrix $D^{*}=\left[d_{ij}^{*}\right]$, I (A*), P is reordered indices of matrix A.

3. Results of Computational Experiment

Computational experiment results of the visual assessment of the clustering tendency based on the VAT and iVAT algorithms with variations of distance measures are presented. The considered distance measures included: Euclidean distance, squared Euclidean distance, standardized Euclidean distance, Manhattan distance, Mahalanobis distance, Bray–Curtis dissimilarity, Canberra distance, Chebyshev distance, distance correlation, cosine similarity.

For the experiments, we used synthetic (artificial) datasets (with cluster labels) [49], as well as from samples of industrial products [50,51] and from a set of some taxation characteristics of forest stands [52]:

a)

Long2 is an artificial dataset that contains the collection of two clusters (1000 data points, 2 dimensions) (Figure 1a).

b)

Sizes1 is an artificial dataset contains the collection of four clusters (1000 data points, 2 dimensions) (Figure 1b).

c)

Longsquare is an artificial dataset contains the collection of six clusters (1000 data points, 2 dimensions) (Figure 1c).

d)

Microchips are two sets of results of test effects on electrical and radio products for monitoring the current–voltage characteristics of input and output circuits of microcircuits: Microchips 140UD25AS1VK (46 data points, 9 dimensions, 2 clusters) and 1526IE10_002 (3987 data points, 67 dimensions, 4 clusters) [50,51]. For microchip 1526IE10_002, we used the following batch combinations: mixed lots from four (62 parameters), three (41 parameters), and two batches (41 parameters).

e)

Siberian Forest Compartments Dataset [52] is a set of some taxation characteristics of forest stands, on which outbreaks of mass reproduction of the Siberian silkworms were recorded at a certain time (15,523 data points, 150 dimensions). The Siberian Forest Compartments Dataset contains three forestry strands: Irbey forestry (2330 data points), Chunsky forestry (3271 data points), and Lower Yenisei forestry (3619 data points).

In our experiments, the following test system was used: Intel (R) Core (TM) i5-8250U CPU, 16 GB RAM, while Python was also used to implement the algorithms.

3.1. Synthetic Datasets

The computational experiment showed that the resulting iVAT images more clearly showed the number of clusters than the resulting VAT images (Appendix A, Figure A1) for Long2. In addition, it can be noted that the Mahalanobis distance and squared Euclidean distance had an advantage over the other distances (Appendix A, Figure A1). The VAT and iVAT algorithms with Canberra distance did not cope with the task. The VAT and iVAT images showed four clusters. The VAT image with Bray–Curtis dissimilarity and cosine similarity showed no clusters (Appendix A, Figure A1).

For the Sizes1 dataset, the computational experiment showed that the resulting iVAT images more clearly showed the number of clusters than the resulting VAT images (Appendix A, Figure A2). In addition, the Canberra distance and cosine similarity had an advantage over the other distances (Appendix A, Figure A2). The VAT and iVAT algorithms with distance correlation did not cope with the task. The VAT and iVAT images showed two clusters. The VAT image with Bray–Curtis dissimilarity showed no clusters (Appendix A, Figure A2).

For Longsquare, the computational experiment showed that the resulting iVAT images more clearly showed the number of clusters than the resulting VAT images (Appendix A, Figure A3). In addition, the Mahalanobis distance and squared Euclidean distance had an advantage over the other distances (Appendix A, Figure A3). The VAT and iVAT algorithms with Bray–Curtis dissimilarity, distance correlation, Canberra distance, and cosine similarity did not cope with the task. The VAT and iVAT images with distance correlation and iVAT images with cosine similarity showed two clusters, while the VAT image with Bray–Curtis dissimilarity showed no clusters (Appendix A, Figure A3).

3.2. Microchips Datasets

For Microchips 140UD25AS1VK, the computational experiment showed that the resulting iVAT images more clearly showed the number of clusters than the resulting VAT images. The VAT and iVAT algorithms with Mahalanobis distance did not cope with the task. The VAT and iVAT images with Mahalanobis distance did not show actual clusters. The iVAT image with distance correlation and cosine similarity showed better performance (Appendix A, Figure A4).

For Microchips 1526IE10_002, four-batch mixed lot, the computational experiment showed that the VAT and iVAT algorithms with Mahalanobis distance, Chebyshev distance, and iVAT with squared Euclidean distance did not cope with the task. The VAT and iVAT images with Mahalanobis distance and the iVAT with squared Euclidean distance showed no clusters (Appendix A, Figure A5).

For Microchips 1526IE10_002, three-batch mixed lot, the computational experiment showed that the resulting iVAT images more clearly showed the number of clusters than the resulting VAT images (Appendix A, Figure A6). The cosine similarity and squared Euclidean distance had an advantage over the other distances (Appendix A, Figure A6). The VAT and iVAT algorithms with Mahalanobis distance and Chebyshev distance did not cope with the task. The VAT and iVAT images with Mahalanobis showed no clusters (Appendix A, Figure A6).

For Microchips 1526IE10_002, two-batch mixed lot, the computational experiment showed that the resulting iVAT images more clearly showed the number of clusters than the resulting VAT images (Appendix A, Figure A7). The VAT and iVAT algorithms with Mahalanobis distance and Chebyshev distance did not cope with the task. The VAT and iVAT images with Mahalanobis showed no clusters (Appendix A, Figure A7).

3.3. Siberian Forest Compartments Datasets

For Chunsky forestry, the computational experiment showed that the resulting iVAT images more clearly showed the number of clusters than the resulting VAT images. However, the Euclidean distance and squared Euclidean distance had an advantage over the other distances (Appendix A, Figure A8).

For Irbey forestry, the computational experiment showed that the resulting iVAT images more clearly showed the number of clusters than the resulting VAT images. The Canberra distance, distance correlation, and cosine similarity had an advantage over the other distances (Appendix A, Figure A9).

For Lower Yenisei forestry, computational experiment showed that the resulting iVAT images more clearly showed the number of clusters than the resulting VAT images (Appendix A, Figure A10).

4. Discussion

Before applying any clustering method, it is important to evaluate cluster validity, i.e., to assess whether the datasets contain meaningful clusters. If clusters exist, then the assessment of the cluster tendency is a good tool to get prior knowledge of the number of clusters in problems such as k-means, where calculating the k value is a challenge.

Despite the fact that there are many similarity measures that have been considered in the literature, it is quite difficult to choose the right one for a specific dataset. Due to the multidimensionality of the data, the cluster structure is not visible explicitly, so the choice of the most suitable distance measure makes it possible to clearly identify this structure. In this research, we modified the VAT and iVAT algorithms in such a way to be able to calculate many possible distance measures besides the Euclidean. The computational experiments showed that using different similarity measures in VAT and iVAT algorithms allows the expert to more confidently estimate the clustering tendency of the dataset.

The computational experiments showed that cluster heat maps produced by the iVAT algorithm in all cases had higher contrast than produced by the VAT algorithm and, hence, were more clear for interpretation.

Our studies have shown that distance measure significantly affects the visual tendency of clustering. In most cases, the best results were obtained with squared Euclidean distance unlike Euclidean distance, which is usually used by default in the VAT family of algorithms.