Cover Tree-Optimized Spectral Clustering: Efficient Nearest Neighbor Search for Large-Scale Data Partitioning

Abstract

Spectral clustering has established itself as a powerful technique for data partitioning across various domains due to its ability to handle complex cluster structures. However, its computational efficiency remains a challenge, especially with large datasets. In this paper, we propose an enhancement of spectral clustering by integrating Cover tree data structure to optimize the nearest neighbor search, a crucial step in the construction of similarity graphs. Cover trees are a type of spatial tree that allow for efficient exact nearest neighbor queries in high-dimensional spaces. By embedding this technique into the spectral clustering framework, we achieve significant reductions in computational cost while maintaining clustering accuracy. Through extensive experiments on random, synthetic, and real-world datasets, we demonstrate that our approach outperforms traditional spectral clustering methods in terms of scalability and execution speed, without compromising the quality of the resultant clusters. This work provides a more efficient utilization of spectral clustering in big data applications.

1. Introduction

Clustering is a foundational unsupervised learning task that aims to partition data points into distinct groups based on similarity. Among existing approaches, spectral clustering has gained significant attention due to its ability to identify non-convex clusters and its solid theoretical foundation in spectral graph theory [1,2]. The core of this technique involves the eigendecomposition of a graph Laplacian matrix to embed data points into a low-dimensional eigenspace where clusters become more separable. For our work, we chose to employ the normalized graph Laplacian $L_{\mathrm{sym}}=I-D^{-1/2}W{D}^{-1/2}$, where W is the similarity matrix and D is the degree matrix [2,3]. This choice is motivated by its well-established theoretical properties and its tendency to yield more stable and consistent clustering results across varied data densities compared to the unnormalized Laplacian.

However, the practical application of spectral clustering to large-scale datasets is severely hampered by its substantial computational complexity. The primary bottlenecks are twofold:

• Similarity Matrix Construction: Building the dense $n\times n$ pairwise similarity matrix W has a prohibitive $O\left(n^2\right)$ time and space complexity.
• Eigendecomposition: Solving the eigenproblem for the Laplacian matrix scales as $O\left(n^3\right)$. While approximate methods can reduce this, the quadratic scaling of constructing W remains the dominant constraint.

To mitigate these challenges, various approximation schemes have been proposed, including sparse k-NN graphs and Nyström methods [4,5]. A highly promising direction involves leveraging advanced data structures for efficient computation. Notably, Cover trees [6] offer a principled framework for accelerating nearest neighbor queries with a query time complexity of $O(c^{12}logn)$ in metric spaces, where c is the expansion constant, a significant improvement over a naive $O\left(n\right)$ linear scan. Beyond mere acceleration, the hierarchical multi-scale structure of Cover trees provides a natural mechanism for intelligent data summarization and noise reduction, making them particularly suitable for enhancing clustering algorithms [7,8].

In this work, we propose the Improved Spectral Clustering with Cover tree (ISCT) algorithm, a novel framework designed to overcome the scalability limitations of traditional spectral clustering. Our approach leverages the Cover tree for a dual purpose within a two-stage approximation framework:

• Data Reduction via Cover tree-Based Summarization: We exploit the hierarchical structure of the Cover tree to select a small set of m representative points (where $m\ll n$) that preserve the essential geometric and density structure of the original dataset. This step drastically reduces the size of the problem.
• Efficient Spectral Clustering on Representatives: We perform spectral clustering using the normalized Laplacian $L_{\mathrm{sym}}$ on the resulting manageable $m\times m$ similarity matrix constructed from the representatives. The cluster labels for all n original points are then accurately inferred by assigning each point to the cluster of its nearest representative, a step executed efficiently in $O(nlogm)$ time using the Cover tree itself for fast nearest-neighbor queries.

The key insight of our approach is this dual use of the Cover tree, which provides not only a fast query structure but also an intelligent, multi-scale data summary that is inherently aligned with the clustering objective. This shifts the computational bottleneck from $O\left(n^3\right)$ to $O(m^3+nlogm)$. Crucially, empirical observations and theoretical bounds suggest that the number of representatives m required for a faithful approximation can favorably scale from $O(logn)$ to $O\left(\sqrt{n}\right)$ for real-world datasets, promising orders-of-magnitude improvement in computational efficiency.

This work is guided by the following research questions:

• How does Cover tree-based data summarization affect the quality of the spectral embedding and the resulting clusters compared to traditional methods, particularly when using the normalized Laplacian?
• What is the computational accuracy trade-off inherent in the ISCT framework, and how is it influenced by the number of representatives m?
• How does the multi-scale hierarchy of the Cover tree influence the detection of cluster boundaries and the separation of clusters at different density levels?

The ISCT framework combines the solid theoretical foundations of spectral clustering, embodied by the normalized Laplacian, with the effective hierarchical properties of cover trees to create a scalable and efficient solution for large-scale clustering challenges.

2. Related Work

The field of cluster analysis has undergone significant evolution, representing one of the earliest developments in data analysis. Its conceptual foundations were established by Driver and Kroeber in 1932 [9], who introduced the fundamental notion of partitioning observations into groups, notably preceding the formal emergence of artificial intelligence by decades. This early work, along with crucial theoretical contributions by Zubin (1938) and Tryon (1939) [10,11] on cluster validity, established the mathematical framework for modern clustering research. The transition to practical algorithmic implementations occurred in the 1960s–1970s with MacQueen’s introduction of the K-Means algorithm, representing a watershed moment that provided the first computationally tractable partition-based approach. Concurrently, hierarchical clustering methods gained prominence through the work of Johnson (1967) [12], while the 1970s witnessed the emergence of probabilistic mixture models. A fundamental paradigm shift occurred with the development of spectral clustering methods, grounded in Fiedler’s work on algebraic connectivity and subsequently refined by Shi and Malik (2000) [3] through normalized cuts. The advent of big data in the 2000s exposed the scalability limitations of traditional methods, particularly the $\mathcal{O}\left(n^3\right)$ eigendecomposition bottleneck in spectral clustering, sparking intensive research into approximate algorithms. This period saw the emergence of Cover trees, providing a rigorous framework for efficient nearest neighbor search. Contemporary developments have focused on addressing the dual challenges of scalability and quality preservation, including reduced clustering methods based on the inversion formula density estimation by Lukauskas et al. (2023) [13] and novel approaches for analyzing research community dynamics by Cambe et al. (2022) [14].

The development of the clustering methodology provides essential context for our proposed algorithm. We focus on the evolution of spectral clustering and related scalability solutions. The theoretical underpinnings of spectral clustering are deeply rooted in spectral graph theory [15]. The algorithm’s effectiveness for non-convex clusters stems from its relaxation of graph partitioning problems, such as normalized cut [3]. The standard algorithm involves constructing a similarity graph, forming a Laplacian matrix (typically $L_{\mathrm{sym}}$ or $L_{\mathrm{rw}}$), computing the first k eigenvectors of the Laplacian and applying a standard clustering algorithm like k-means to the rows of the eigenvector matrix to obtain the final partition. The phrase “simplified bisection via the eigenvector” often refers to the special case of $k=2$, where the second eigenvector (the Fiedler vector) is thresholded to partition the graph [16]. For $k>2$, the process is a generalization of this bisection concept [13].

The high computational cost of spectral clustering has spurred research into approximations. A common approach is to sparsify the similarity matrix using k-NN or $ϵ$-neighborhood graphs, reducing the storage to $O\left(nk\right)$ and enabling the use of sparse eigensolvers like ARPACK [17]. The Nyström method is another prominent technique, which approximates the eigendecomposition by using a subset of landmark points [5]. The complexity is reduced to $O(n{l}^2+l^3)$ for l landmarks. Our method differs by using the Cover tree not just to select landmarks but to ensure they provide a geometrically meaningful summary of the entire dataset [14].

Data structures like KD-Trees, Ball Trees, and Cover trees have been widely used to accelerate distance-based computations. Their integration into spectral clustering has primarily been limited to constructing approximate k-NN graphs [18]. The ISCT algorithm extends this integration beyond a mere neighbor search to a holistic data reduction and assignment strategy, leveraging the Cover tree’s theoretical guarantees for both tasks.

Our proposed ISCT algorithm represents a natural evolution of this historical trajectory, integrating Cover tree hierarchical decomposition with spectral techniques to bridge the gap between classical graph-theoretic approaches and modern scalable computing requirements, and building upon this rich methodological heritage to address specific computational bottlenecks that limit spectral methods’ applicability to large-scale datasets.

3. Methods

3.1. Spectral Clustring

Spectral clustering is a sophisticated and widely-used technique in the field of machine learning and data analysis for partitioning data into clusters based on the eigenvalues (spectrum) of a similarity matrix derived from the data [19]. Unlike traditional clustering methods such as K-Means, which rely on distance metrics to form clusters, spectral clustering leverages the connectivity structure of the data [20]. The process begins with the construction of a similarity graph $G=(V,E)$, where nodes represent data points and edges represent the similarity between pairs of points. This graph is then transformed into a Laplacian matrix, whose eigenvectors are used to map the data points into a lower-dimensional space.

The adjacency matrix A of the graph is defined as follows:

$$A_{ij}=\left\{\begin{array}{cc}\mathrm{similarity}(i,j)\hfill & \mathrm{if}(i,j)\in E,\hfill \\ 0\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

The degree matrix D is a diagonal matrix where each diagonal element $D_{ii}$ is the sum of the similarities of a node i:

$$D_{ii}=\sum _j{A}_{ij}.$$

The unnormalized graph Laplacian L is then given by

$$L=D-A.$$

Alternatively, the normalized graph Laplacian can be defined in two forms:

$$L_{\mathrm{sym}}=I-D^{-1/2}A{D}^{-1/2},$$

$$L_{\mathrm{rw}}=I-D^{-1}A.$$

The next step involves computing the eigenvalues and eigenvectors of the Laplacian matrix. Let ${\lambda }_1,{\lambda }_2,\dots ,{\lambda }_n$ be the eigenvalues and $v_1,v_2,\dots ,v_n$ be the corresponding eigenvectors. By selecting the eigenvectors corresponding to the smallest k eigenvalues (excluding the zero eigenvalue for the normalized Laplacian) [21], we form a matrix U where each row represents a data point in the new lower-dimensional space:

$$U=[v_1,v_2,\dots ,v_k].$$

Finally, a clustering algorithm such as K-Means is applied to the rows of U to partition the data into k clusters. This method excels in identifying clusters that are not necessarily spherical in shape and can capture complex structures in the data, making it particularly effective for applications such as image segmentation [22], social network analysis, and bioinformatics. The power of spectral clustering lies in its ability to utilize the global information of the data structure [23], providing more accurate and meaningful clusters in scenarios where traditional methods might fail.

3.2. Cover Tree

The Cover tree is a sophisticated data structure specifically designed for facilitating efficient nearest neighbor searches in high-dimensional metric spaces. Introduced by Alina et al. in 2006 [6], the Cover tree addresses the challenge of the “curse of dimensionality” that significantly impacts the performance of traditional nearest neighbor search algorithms as the dimensionality of the data increases. The fundamental principle behind the Cover tree is to maintain multiple hierarchical levels of the dataset, where each level is a “cover” for the level below it [24], ensuring that every point at a lower level is within a certain distance of a point in the level above.

The Cover tree is constructed based on three main properties: nesting, covering, and separation.

Let C be the level and i the index of this level i; the principal equations associated with the Cover tree are defined as follows:

• Nesting property:

  $$C_i\subseteq C_{i-1}.$$
• Cover property: For any point $c\in C_{i-1}$, there exists a point $p\in C_i$ such that:

  $$d(p,c)\le 2^i.$$
• Separation property: For all $p,q\in C_i$ with $p\ne q$:

  $$d(p,q)>2^i.$$
• Parent–child distance: For each child c of a parent p on the scale i:

  $$d(p,c)\le 2^i.$$

One of the key advantages of the Cover tree is its ability to perform nearest neighbor searches in $O(logn)$ time, where n is the number of points in the dataset, making it exceptionally efficient compared to brute-force methods, especially in high-dimensional spaces. The hierarchical structure of the Cover tree allows for a significant reduction in the search space at each level [25], enabling rapid convergence to the nearest neighbor or neighbors.

The Cover tree’s efficiency and scalability make it a valuable tool in various applications, including machine learning, pattern recognition, and data mining, where quick and accurate nearest neighbor searches are crucial. Its design elegantly balances the trade-off between space and time complexity, providing a robust solution to one of the most pervasive challenges in computational geometry and high-dimensional data analysis.

3.3. Proposed Method

The Improved Clustering with Cover tree algorithm aims to efficiently cluster a dataset by leveraging a Cover tree structure. Initially, the algorithm takes as input a data matrix V and the number of desired clusters k, and outputs k clusters. The process begins by decomposing the dataset V using a cover tree [26], which results in m representative points and a tree structure T. This decomposition aids in managing the data more effectively. Next, a similarity matrix W for the representative points is formed. The algorithm then initializes a level variable l to the initial level $l_{\mathrm{init}}$ of the Cover tree [26]. It iteratively processes each level from $l_{\mathrm{init}}$ down to 1, updating the clustering indicators. At the initial level, the indicator matrix ${\Gamma }_l$ is set to an identity matrix or an equivalent initial clustering indicator. For other levels, ${\Gamma }_l$ is computed based on a specific equation, likely involving the similarity matrix W and the tree structure. The level l is decremented after processing all nodes at the current level. Once all levels are processed [27], the algorithm recovers the cluster membership for each original data point by mapping the results from the representative points back to the original dataset. Finally, the algorithm uses the K-Means algorithm to cluster the points in a reduced k-dimensional space, represented by the rows of matrix $C_p$, into k clusters. The algorithm concludes by outputting the k clusters [28], providing the final clustering result. This method enhances efficiency and scalability [23], making it suitable for large and high-dimensional datasets by focusing on representative points and iteratively refining the clusters.

3.3.1. Laplacian Matrix Formulations and Their Implications for ISCT

The choice of Laplacian matrix formulation is a critical design decision in spectral clustering. Given a similarity matrix $W\in {\mathbb{R}}^{m\times m}$ and degree matrix D where $D_{ii}={\sum }_{j=1}^m{W}_{ij}$, the three primary formulations are the unnormalized Laplacian $L=D-W$ preserves cluster size but suffers from scale sensitivity, favoring larger clusters. The symmetric normalized Laplacian $L_{sym}=D^{-1/2}L{D}^{-1/2}=I-D^{-1/2}W{D}^{-1/2}$ addresses scale sensitivity and provides better numerical stability, with eigenvalues satisfying $0\le {\lambda }_i\le 2$. Its eigenvectors require transformation: $v=D^{-1/2}u$ where u are the eigenvectors of $L_{sym}$. And The random walk Laplacian $L_{rw}=D^{-1}L=I-D^{-1}W$ relates to Markov chains but is asymmetric, complicating eigendecomposition. The ISCT algorithm employs $L_{sym}$ for numerical stability with varying node degrees, scale invariance across density regions, computational efficiency from symmetric eigensolvers, and theoretical guarantees related to the Cheeger inequality. The eigendecomposition computes $L_{sym}{u}_i={\lambda }_i{u}_i$ for $i=1,\dots ,k$, with transformed eigenvectors $v_i=D^{-1/2}{u}_i$ used for clustering. The computational complexity involves: computing $D^{-1/2}$ in $O\left(m\right)$ operations, eigendecomposition in $O\left(m^3\right)$ with efficient symmetric solvers, and eigenvector transformation adding $O\left(km\right)$ operations.

Our general Algorithm 1 is as follows:

Algorithm 1 Improved Clustering with Cover tree

1: Input: A data matrix $V\in {\mathbb{R}}^{n\times d}$, the number of clusters k.   2: Output: k clusters   3: Decompose the dataset V using Cover tree:   4: Decompose dataset V by Cover tree to obtain m representative points and tree structure T.   5: Form the representative points similarity matrix:   6: $W\in {\mathbb{R}}^{m\times m}$.   7: Initialize:   8: $l=l_{\mathrm{init}}$;   9:  while $l\ge 1$do 10:      for each node at level l do 11:             if $l=l_{\mathrm{init}}$ then 12:                Set the indicator matrix of the clustering result of current node ${\Gamma }_l=E$; 13:             else 14:                   Compute ${\Gamma }_l$ according to ${\Gamma }^{l-1}={\Gamma }^l{\Gamma }^{\prime l-1}$ where ${\Gamma }^{\prime l-1}$ is the indicator matrix of clusters ; 15:            end if 16:      end for 17:      $l\leftarrow l-1$; 18: end while 19: Recover the cluster membership: 20: To obtain the indicator matrix of the original dataset, recover the cluster membership for each point by looking up the cluster membership of the corresponding superpoints in the correspondence table. 21: Final Clustering: 22: Treat each row of $C_p$ as a point in ${\mathbb{R}}^k$ and cluster all of the points into k clusters via the K-Means algorithm to obtain the final clusters of V. 23: Output: k clusters

To clarify the essential points of our algorithm, the matrix ${\Gamma }^{\prime l-1}$ is defined as a binary matrix representing the hierarchical correspondence of superpoints. Each entry ${\Gamma }^{\prime l-1}(i,j)=1$ indicates that superpoint j at level $l-1$ is associated with superpoint i at level l. Formally, let $C_l$ be the set of superpoints at level l; then,

$${\Gamma }^{\prime l-1}(i,j)=\left\{\begin{array}{cc}1\hfill & \mathrm{if}{c}_j^{l-1}\in \mathrm{children}\left(c_i^l\right),\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

where $\mathrm{children}\left(c_i^l\right)$ denotes the direct subnodes of $c_i^l$ in the tree. This structure ensures consistent clustering between adjacent levels.

The initial level $l_{\mathrm{init}}$ is determined from the geometric properties of the Cover tree. Concretely, $l_{\mathrm{init}}=l_{\max }$, where $l_{\max }$ is the coarsest level, characterized by a maximum node diameter ${\Delta }_{l_{\max }}$ corresponding to the greatest distance between points in the dataset. This diameter is calculated by

$${\Delta }_l=\underset{c\in C_l}{max}\underset{x,y\in c}{max}|x-y|.$$

This calculation ensures that the hierarchical decomposition begins with a global view of the data.

Finally, initializing the E matrix, which corresponds to the ${\Gamma }^{\prime l}$ matrix when $l=l_{\mathrm{init}}$, is completed as follows:

• At the $l_{\mathrm{init}}$ level, each superpoint is treated as a singleton cluster, resulting in $E={\mathbf{I}}_m$ (identity matrix of size $m\times m$), where m is the number of superpoints.

In this paragraph, we will go into more detail about the algorithm presented above. It introduces an innovative approach to clustering large-scale or high-dimensional datasets by leveraging the hierarchical structure of a Cover tree for data decomposition. At its core, the method begins by transforming the input data matrix, denoted as $V\in {\mathbb{R}}^{n\times d}$, where represents the number of data points and the dimensionality of each point into a more manageable form through the construction of a Cover tree [24]. This tree effectively summarizes the dataset by identifying representative points, thereby reducing the complexity of the dataset while preserving its intrinsic structure.

Following the decomposition, a similarity matrix $W\in {\mathbb{R}}^{n\times m}$ is constructed to encapsulate the pairwise similarities between the representative points. This matrix plays a pivotal role in the subsequent clustering process, which is performed iteratively across different levels of the Cover tree. The iterative process begins at a predefined initial level and progresses downwards, with each level’s clustering results influencing the computation of the indicator matrix ${\Gamma }_t$ for the nodes at that level [26]. This hierarchical clustering mechanism ensures that the algorithm efficiently captures the multi-scale structure of the data.

A crucial step in the algorithm involves the recovery of cluster memberships for the original data points from the clustering results of the representative points. This step is facilitated by a correspondence table that maps each data point to its nearest representative point, thereby allowing the algorithm to extend the clustering results from the representative points to the entire dataset.

The final phase of the algorithm employs the K-Means clustering algorithm to refine the clustering results. Each row of the indicator matrix $C_p$, which represents the cluster memberships of the representative points, is treated as a point in $R^d$ [29]. The K-Means algorithm is then applied to these points to produce the final clustering of the original dataset into k clusters.

This sophisticated algorithm harnesses the efficiency of the Cover tree structure to address the challenges associated with clustering large or high-dimensional datasets [30]. By focusing on representative points and employing a hierarchical clustering strategy [31], the algorithm significantly reduces computational complexity while ensuring that the final clustering results accurately reflect the underlying structure of the data.

3.3.2. Mathematical Formalization of ISCT

The method performs hierarchical spectral clustering on a data matrix $V\in {\mathbb{R}}^{n\times d}$ using a Cover tree structure. The process proceeds from finer to coarser levels, where clustering results from child nodes are treated as superpoints for parent nodes. For a parent node $S^{l-1}$ with superpoints $A_{S^{l-1}}=\{A_1^l,A_2^l,\dots ,A_L^l\}$ and child nodes $S_1^l=\{A_1^l,\dots ,A_{k_1}^l\}$, $S_2^l=\{A_{k_1+1}^l,\dots ,A_{k_2}^l\}$ where $S^{l-1}=S_1^l\cup S_2^l$, the indicator matrices for the child clustering results $C_1^l=\{C_{1,1}^l,\dots ,C_{1,o_1}^l\}$ and $C_2^l=\{C_{2,1}^l,\dots ,C_{2,o_2}^l\}$ are defined as follows:

$${\Gamma }_1^l(i,j)=\left\{\begin{array}{cc}{\displaystyle \frac{1}{\sqrt{|C_{1,j}^l|}}},\hfill & \mathrm{if}{A}_i^l\in C_{1,j}^l,\hfill \\ 0,\hfill & \mathrm{otherwise},\hfill \end{array}\right.(i=1,\dots ,k_1;j=1,\dots ,o_1)$$

with ${\Gamma }_2^l$ defined similarly. These are concatenated into

$${\Gamma }^l=\left[\begin{array}{c}{\Gamma }_1^l\\ {\Gamma }_2^l\end{array}\right]$$

The reduced similarity matrix for the superpoint set $C^l={\bigcup }_{i=1}^2{C}_i^l$ is computed as:

$${\tilde{W}}_{S^{l-1}}={\left({\Gamma }^l\right)}^{\top }{W}_{S^{l-1}}{\Gamma }^l$$

where $W_{S^{l-1}}$ is the principal submatrix of the original similarity matrix W corresponding to $S^{l-1}$.

The clustering problem is formulated as the following minimization:

$$\begin{array}{cc}\hfill \underset{{\Gamma }_{1:l}^l}{min}& Tr\left({\left({\Gamma }_{1:l}^l\right)}^{\top }{\tilde{L}}_{S^{l-1}}{\Gamma }_{1:l}^l\right),\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& {\left({\Gamma }_{1:l}^l\right)}^{\top }{\Gamma }_{1:l}^l=I,\hfill \end{array}$$

where ${\tilde{L}}_{S^{l-1}}={\tilde{D}}_{S^{l-1}}-{\tilde{W}}_{S^{l-1}}$ is the Laplacian matrix. The solution is given by the first k eigenvectors of ${\tilde{L}}_{S^{l-1}}$.

The indicator matrix for the original superpoints $A_{S^{l-1}}$ is obtained through

$${\Gamma }^{l-1}={\Gamma }^l{\Gamma }_{1:l}^l$$

This process repeats up the Cover tree hierarchy until the final indicator matrix ${\Gamma }^1$ is obtained. The cluster membership for original data points is recovered via a correspondence table, yielding the indicator matrix $C_p$. Finally, each row of $C_p$ is treated as a point in ${\mathbb{R}}^k$ and clustered via k-means to produce the final result. The key advantage is the significant computational complexity reduction achieved by operating on the much smaller matrices ${\tilde{W}}_{S^{l-1}}$ and ${\tilde{L}}_{S^{l-1}}$ instead of the full matrices at each level.

3.3.3. Similarity Definition and Graph Construction

The algorithm constructs a similarity matrix $W\in {\mathbb{R}}^{m\times m}$ on m representative points (where $m\ll n$) from a Cover tree decomposition, rather than on all n data points, significantly reducing computational complexity while maintaining clustering quality.

The similarity measurement begins with the Euclidean distance metric:

$$d(x_i,x_j)={\parallel x_i-x_j\parallel }_2=\sqrt{\sum _{k=1}^d{(x_i^k-x_j^k)}^2}$$

which is then converted to similarity using the Gaussian radial basis function (RBF) kernel:

$$w(x_i,x_j)=exp\left(-{\displaystyle \frac{d{(x_i,x_j)}^2}{2{\sigma }^2}}\right)$$

where $\sigma >0$ is the bandwidth parameter controlling similarity decay.

The complete similarity matrix construction is formalized in the following Algorithm 2.

Algorithm 2 Similarity Matrix Construction

Input : m representative points from Cover tree decomposition Output: Sparse symmetric similarity matrix $W\in {\mathbb{R}}^{m\times m}$ Standardize representative points to ensure unit variance across dimensions for each point $i\in \{1,\dots ,m\}$ do       Query Cover tree for k nearest neighbors using Euclidean distance       Compute RBF weights: $w_{ij}=exp(-\parallel x_i-x_j{\parallel }_2^2/\left(2{\sigma }^2\right))$ for each neighbor j end for Construct symmetric matrix: $W={\displaystyle \frac{1}{2}}(W+W^T)$

This approach leverages the Cover tree’s logarithmic query complexity for efficient k-NN graph construction while maintaining the local geometric structure necessary for effective spectral clustering. The resulting sparse, symmetric matrix W captures meaningful similarity relationships between representative points, forming the foundation for subsequent spectral analysis.

3.3.4. Analysis of Parameter Sensitivity and Selection Methodology

The proposed Cover tree-accelerated spectral clustering framework involves four key hyperparameters that balance computational efficiency and clustering fidelity. In the following, we will introduce how we determined the number of representative points m, neighborhood size $k_{nn}$, kernel scale $\sigma$, and number of clusters k.

The number of representatives m is selected through a hybrid approach: heuristic initialization $m=min(5000,\sqrt{n})$ followed by sensitivity analysis to find the smallest m that maintains clustering quality.

For $k_{nn}$ and $\sigma$ parameters, an adaptive Gaussian kernel is employed. The similarity between points is calculated using the following equation:

$$w_{ij}=exp\left(-{\displaystyle \frac{d{(x_i,x_j)}^2}{{\sigma }_i·{\sigma }_j}}\right)$$

where ${\sigma }_i$ is the distance to the $k_{nn}$-th nearest neighbor of $x_i$, reducing parameter tuning to selecting $k_{nn}$ via grid search.

The number of clusters k is determined through a hierarchical validation framework evaluating candidate values:

$$k_{candidates}=\{2,3,\dots ,\min (\mathrm{nodes}\_\mathrm{at}\_\mathrm{level}(l),k_{max})\}$$

with $k_{max}=\sqrt{m}$. For each candidate k, a composite score is computed through combining three validation metrics:

The relative eigenvalue gap:

$${\mathrm{RelativeGap}}_k={\displaystyle \frac{{\lambda }_{k+1}-{\lambda }_k}{{\lambda }_{k+1}}}$$

The modularity measure:

$$Q={\displaystyle \frac{1}{2m}}\sum _{i,j}\left[W_{ij}-{\displaystyle \frac{d_i{d}_j}{2m}}\right]\delta (c_i,c_j)$$

The silhouette coefficient:

$$s\left(i\right)={\displaystyle \frac{b\left(i\right)-a\left(i\right)}{\max \left\{a\right(i),b(i\left)\right\}}}$$

The cluster selection Algorithm 3 proceeds through hierarchical evaluation of these metrics.

Algorithm 3 Adaptive Cluster Number Selection in ISCT

1: Input: Cover tree levels $\{1,2,\dots ,l_{max}\}$, similarity matrix W   2: Output: Optimal cluster number $k^{\ast }$   3: for each level l in Cover tree do   4:       Compute eigenvalues $\{{\lambda }_1,\dots ,{\lambda }_m\}$ of $L_{sym}$   5:       for $k=2$ to $min(\mathrm{nodes}\_\mathrm{at}\_\mathrm{level}\left(l\right),k_{max})$ do   6:              Compute ${\mathrm{RelativeGap}}_k$   7:              Perform spectral clustering with k clusters   8:              Calculate modularity $Q_k$ and silhouette score $S_k$   9:              Compute composite score: ${\mathrm{Score}}_k=\alpha ·{\mathrm{RelativeGap}}_k+\beta ·Q_k+\gamma ·S_k$ 10:       end for 11:       Select $k_l^{*}=arg{max}_k{\mathrm{Score}}_k$ 12: end for 13: Return: $k^{\ast }=\mathrm{majority}\_\mathrm{vote}\left(\{k_1^{\ast },k_2^{\ast },\dots ,k_{l_{max}}^{\ast }\}\right)$

With typical weights $\alpha =0.4$, $\beta =0.3$, $\gamma =0.3$. Cluster stability is assessed using the following stability metric:

$$\mathrm{Stability}=1-{\displaystyle \frac{\mathrm{var}\left(\{k_1^{\ast },k_2^{\ast },\dots ,k_{l_{max}}^{\ast }\}\right)}{\mathrm{mean}\left(\{k_1^{\ast },k_2^{\ast },\dots ,k_{l_{max}}^{\ast }\}\right)}}$$

where high stability values (>0.8) indicate robust cluster number detection. This comprehensive approach addresses the fundamental challenge of determining appropriate cluster numbers in unsupervised settings.

4. Theoretical Foundations for Accuracy Preservation

A central claim of our Cover tree-accelerated spectral clustering approach is its ability to maintain high clustering accuracy despite operating on a significantly reduced set of $m\ll n$ representative points. This section delineates the theoretical underpinnings that justify this claim, explaining how the algorithm’s design inherently preserves the complex geometric structures present in the original data.

4.1. Preservation of Local Geometry by the Cover Tree

The Cover tree decomposition provides a hierarchical net that faithfully captures dataset topology through its nesting and covering invariants, offering distinct advantages over random sampling methods.

The Cover tree decomposition ensures data representation fidelity through two fundamental properties. It maintains exact point representation, where each node corresponds to an actual data point from the original set V rather than centroids or averages, thereby preserving complex non-linear features without distortion. Additionally, it provides guaranteed proximity by ensuring that for any point $v_i\in V$, there exists a representative within a precisely defined distance at every tree level, forming a dense hierarchical skeleton that captures the complete dataset structure.

Here is an Algorithm 4 that explains this process in more detail.

Algorithm 4 Hierarchical Spectral Clustering on Cover tree

1: Build Cover tree T from dataset V 2: Let $l_{\max }$ be the top level of T 3: Perform spectral clustering on representatives at level $l_{\max }$ to get ${\Gamma }^{l_{\max }}$ 4: for $l=l_{\max }-1$ downto 1 do 5:       Propagate: Compute clustering ${\Gamma }^l$ informed by coarser clustering ${\Gamma }^{l+1}$ 6:       ${\Gamma }^l\leftarrow \mathrm{RefineClustering}({\Gamma }^{l+1},\mathrm{Representatives}\mathrm{at}\mathrm{level}l)$ 7: end for 8: Recover full labels for V via correspondence lookup

Final Label Assignment is defined by

$$\mathrm{label}\left(v_i\right)=\mathrm{label}\left(\mathrm{rep}\left(v_i\right)\right)$$

where $\mathrm{rep}\left(v_i\right)$ is the representative point for $v_i$.

The Role of the Final K-Means Step

The K-Means step clusters the rows of the eigenvector matrix U to resolve the final partitioning in the transformed spectral embedding space, where clusters are linearly separable, providing a clean, hard assignment based on the complex cluster boundaries identified by the hierarchical spectral clustering process.

Enhancing Robustness and Noise Resilience

To enhance robustness against noise and outliers, two refined graph construction methods are employed. The Mutual k-NN (MkNN) method establishes an edge between points only if they are mutually included in each other’s k-nearest neighbor sets, ensuring reciprocal connectivity. The $\epsilon$-k-NN Hybrid method further requires that the distance between points remains below a threshold $\epsilon$, combining both relative and absolute proximity criteria. Together, these techniques generate a sparser and more reliable similarity graph that minimizes the impact of outliers while faithfully representing the true underlying cluster structure.

4.2. Leveraging the Cover Tree for Proactive Outlier Detection

The Cover tree hierarchy inherently reveals data density, enabling proactive outlier identification. Representatives with few associated data points are flagged as potential outliers using a density threshold ${\tau }_{\mathrm{density}}$:

• If $c_i<{\tau }_{\mathrm{density}}\Rightarrow$ representative i is flagged as PotentialNoise

The Algorithm 5 for this task can be summarized as follows:

Algorithm 5 Integrated Robust Spectral Clustering

1: Build Cover tree and compute point counts $c_i$ for each representative 2: Flag representatives with $c_i<{\tau }_{\mathrm{density}}$ as PotentialNoise 3: Construct Mutual k-NN graph W excluding noise representatives 4: Identify connected components and flag small components (<${\tau }_{\mathrm{component}}$) as Noise 5: Perform spectral clustering on remaining “core” representatives 6: Assign labels to original points: $\mathrm{label}\left(v_j\right)=\mathrm{label}\left(\mathrm{rep}\left(v_j\right)\right)$ for non-noise representatives 7: Assign $\mathrm{label}\left(v_j\right)=-1$ for points represented by noise representatives

This is a summary of parametrization.

Parameter	Purpose
${\tau }_{\mathrm{density}}$	Minimum point count for dense region classification
$k_{\mathrm{mutual}}$	Neighborhood size for Mutual k-NN graph
${\tau }_{\mathrm{component}}$	Minimum size for valid cluster components

This integrated approach leverages the Cover tree structure for proactive noise identification and removal, enhancing clustering robustness without significant computational overhead. The method combines density-based outlier detection with graph-theoretic noise filtering before performing spectral clustering on the purified dataset.

5. Materials and Environment

For our research, we utilized an ASUS ROG Strix computer featuring an Intel(R) Core(TM) i7-10870H processor with a base clock speed of 2.20 GHz, capable of reaching 2.21 GHz, paired with 16 GB of RAM and an NVIDIA RTX 2060 graphics card. The computer was sourced from ASUSTeK Computer Inc., Taipei, Taiwan. The processor was sourced from Intel Corporation, Santa Clara, California, United States. The graphics card was sourced from NVIDIA Corporation, Santa Clara, California, United States. This hardware was selected for its capacity to handle the demanding computational tasks typically required for data science modeling and simulations. We conducted our experiments using Python version 3.12 environment provided by Anaconda, deliberately avoiding the integration of any additional packages. This minimalist setup was chosen to develop our model and ensure that no external features influenced its performance. Conversely, for the alternative models we analyzed, additional packages included in the Anaconda distribution were utilized. These models leveraged specialized libraries, which allowed for performance optimizations and reduced development time. This approach enabled a comparative analysis between our streamlined model, operating in a pure Python environment, and the alternative models enhanced by advanced libraries. The findings underscored the efficiency of our simplified model in terms of performance.

6. Results

To compare our algorithm with others, the real datasets chosen were Iris, Seeds, Glass, Mall, Cancer, as well as synthetic data such as Blobs, Circles and Moons, which are represented in the Table 1 and the other real datasets are represented in Table 2, as they are frequently used for testing and evaluating machine learning algorithms; the use of synthetic datasets such as Three Moons, Blobs and Circles is a particularly common practice for evaluating and visualizing the performance of clustering algorithms. These datasets are specifically designed to test the ability of clustering methods to identify and separate distinct groups in data of varying complexity and structures. Three Moons, with its three intertwined crescent shapes, represents a challenge for algorithms due to its non-convex shape. Blobs contain well-separated Gaussian clusters to evaluate clustering on spherical groups. Circles feature points arranged in concentric circles to test the handling of nested and non-linearly separable clusters.

Table 1. Description of real and synthetic datasets used.

Dataset	Instances	Clusters	Dimensions	Reference
Iris	150	3	4	[32]
Seed	210	3	7	[33]
Glass	214	3	10	[34]
Mall	200	5	5	[35]
Breast Cancer	569	2	30	[36]
Blobs Data	500	5	2	–
Circles Data	500	5	2	–
Moons Data	500	5	2	–

Table 2. Description of higher dimensions datasets.

Dataset	Sample Size (N)	Dimensions (D)	Classes	Citation
Arrhythmia	452	262	13	[37]
Coil20	1440	1024	20	[38]
Dermatology	366	17	6	[39]
German Credit	1000	60	2	[40]
Heart-statlog	270	13	2	[41]
Ionosphere	351	34	2	[42]
Segment	2310	19	7	[13]
Spambase	4601	57	2	[43]
WDBC	569	30	2	[44]

We evaluated the clustering algorithms on three synthetic benchmark datasets: two interleaving half-moons (Moons) and concentric circles (Circles), and a Gaussian Blobs dataset. The Moons and Circles datasets each contained 500 points with two ground-truth classes, while the Blobs dataset comprised 500 points distributed across five Gaussian clusters. To ensure reproducibility, we generated the data using Scikit-learn’s dataset generators with fixed parameters: Moons with n_samples = 500, noise = 0.1, and random_state = 40; Circles with n_samples = 500, noise = 0.1, factor = 0.1, and random_state = 40; and Blobs with n_samples = 500, centers = 5, cluster_std = 3.0, and random_state = 40.

All clustering algorithms were configured to identify $k=5$ clusters. This setup intentionally creates a mismatch between the true number of classes 2 and the number of predicted clusters 5 for the Moons and Circles datasets. To address this discrepancy, we employed a many-to-one relabeling strategy (also known as cluster purity mapping). Specifically, each predicted cluster was assigned to the ground-truth class that represents the majority of its points. The mapping was established by finding the mode of the true labels within each cluster using SciPy’s mode function. This alignment allowed us to compute standard supervised evaluation metrics, including Adjusted Rand Index (ARI), Normalized Mutual Information (NMI), and classification-style metrics (Precision, Recall, F1-score). Additionally, we reported internal clustering quality measures such as the Silhouette score. A many-to-one confusion matrix was also constructed to visualize the distribution of true classes across predicted clusters.

The three figures Figure 1, Figure 2 and Figure 3 illustrate the dataset distribution and the selected representative points for each clustering configuration. In all visualizations, grey circles represent the original data points prior to clustering, while red crosses designate the representative points (marked by red crosses) selected by the Cover Tree in the ISCT algorithm are illustrated for three standard synthetic datasets: Moons, Circles, and Blobs. These representatives provide a multi-scale summarization of each dataset. Only a small subset of the original points was retained as representatives, yielding a compact core set that significantly reduced the computational load of subsequent clustering while preserving the intrinsic topology of the data. Because the Cover Tree is hierarchical, the red-cross nodes correspond to points chosen at intermediate levels of the decomposition, reflecting a coarse-to-fine grouping of the data. These visualizations thus demonstrate how ISCT approximates the overall dataset structure via this sparse representation prior to the final clustering stage.

Figure 1. Representative points on Circles data.

Figure 2. Representative points on Moons data.

Figure 3. Representative points on Blobs data.

We also used various random datasets represented in the Table 3 to analyze and evaluate the behavior and performance of clustering algorithms across different distributions and dimensions. The datasets generated include a wide range of distributions and parameters, ensuring a comprehensive evaluation. Each dataset enabled clustering methodologies to be tested and validated in a variety of scenarios. These datasets were generated using NumPy’s random module, encompassing both continuous and discrete probability distributions. Continuous data was produced using uniform (np.random.rand), normal (np.random.normal), exponential (np.random.exponential), gamma (np.random.gamma), and log-normal (np.random.lognormal) distributions. Discrete data included integer uniform (np.random.randint), binomial (np.random.binomial), Poisson (np.random.poisson), and Bernoulli distributions. All distributions were parameterized according to their statistical properties, and random seed initialization ensured the reproducibility of the generated data sequences for simulation and testing purposes.

Table 3. Random datasets of different distributions used.

Dataset	Description	Instances	Dimensions	Clusters
Random Data	Range: 0–100	100	20	3
Integer Data	Range: 0–100	500	20	4
Normal Data	$\mu =0$, $\sigma =1$	5000	10	3
Exponential Data	$\lambda =0.5$	1000	20	3
Uniform Discrete Data	Range: 0–9	1500	10	2
Binomial Data	$n=10$, $p=0.5$	5000	20	3
Gamma Data	Shape: 1, Scale: 2	700	10	2
Lognormal Data	$\mu =0$, $\sigma =1$	2000	10	3
Poisson Data	$\lambda =2$	4200	15	3
Bernoulli Data	$p=0.3$	7000	10	4

To evaluate and compare the effectiveness of the ISCT (Improved Clustering with Cover tree), Spectral Clustering and K-Means clustering algorithms, three commonly used evaluation indices are mobilized: the Silhouette index, the Davies–Bouldin index and the Calinski–Harabasz index.

The Silhouette Index measures how similar an object is to its own cluster compared to other clusters. It ranges from −1 to +1, where a high value indicates that the object is well matched to its own cluster and poorly matched to neighboring clusters.

For a data point i, the silhouette score $SI\left(j\right)$ is defined as follows:

$$SI\left(j\right)={\displaystyle \frac{b\left(j\right)-a\left(j\right)}{max\left(a\right(j),b(j\left)\right)}},$$

where $a\left(j\right)$ is the average distance between j and all other points in the same cluster. and $b\left(j\right)$ is the minimum average distance from j to all points in other clusters.

The Davies–Bouldin Index (DBI) is used to evaluate clustering algorithms. A lower DBI indicates better clustering. It is defined as the average similarity ratio of each cluster with the cluster that is most similar to it; a lower value indicates better clustering based on intra-cluster and inter-cluster distances.

For k clusters, the DBI is

$$DBI={\displaystyle \frac{1}{k}}\sum _{i=1}^k\underset{j\ne i}{max}\left({\displaystyle \frac{s_i+s_j}{d_{ij}}}\right),$$

where $s_i$ is the average distance between each point in cluster i and the centroid of cluster i. $d_{ij}$ is the distance between the centroids of clusters i and j.

The Calinski–Harabasz Index (also known as the Variance Ratio Criterion) evaluates the ratio of the sum of between-cluster dispersion and within-cluster dispersion. A higher CH index indicates better-defined clusters; a higher value indicates better clustering based on the ratio of between-cluster dispersion to within-cluster dispersion.

For k clusters, the CH index is defined as follows:

$$CH={\displaystyle \frac{\mathrm{Tr}\left(B_k\right)/(k-1)}{\mathrm{Tr}\left(W_k\right)/(n-k)}},$$

where $\mathrm{Tr}\left(B_k\right)$ is the trace of the between-group dispersion matrix; $\mathrm{Tr}\left(W_k\right)$ is the trace of the within-cluster dispersion matrix; n is the total number of points; and k is the number of clusters. Silhouette is indicated by SI, Davies by Da, and Calinski by Ca.

In unsupervised learning, evaluating partition quality is crucial. Two notable metrics are the Adjusted Rand Index (ARI), denoted $\rho$, and the Normalized Mutual Information (NMI), denoted $\nu$.

The ARI $\rho$ corrects for chance in partition similarity, where a value of 1 indicates perfect agreement. It is calculated as follows:

$$\rho ={\displaystyle \frac{{\sum }_{ij}\left(\genfrac{}{}{0pt}{}{n_{ij}}{2}\right)-\frac{\left[{\sum }_i\left(\genfrac{}{}{0pt}{}{a_i}{2}\right){\sum }_j\left(\genfrac{}{}{0pt}{}{b_j}{2}\right)\right]}{\left(\genfrac{}{}{0pt}{}{n}{2}\right)}}{\frac{1}{2}\left[{\sum }_i\left(\genfrac{}{}{0pt}{}{a_i}{2}\right)+{\sum }_j\left(\genfrac{}{}{0pt}{}{b_j}{2}\right)\right]-\frac{\left[{\sum }_i\left(\genfrac{}{}{0pt}{}{a_i}{2}\right){\sum }_j\left(\genfrac{}{}{0pt}{}{b_j}{2}\right)\right]}{\left(\genfrac{}{}{0pt}{}{n}{2}\right)}}}$$

where $n_{ij}$ is an element of the contingency table, $a_i$ and $b_j$ are row and column sums, and n is the total number of observations.

The NMI $\nu$ measures normalized informational redundancy between labelings, with $\nu =1$ indicating perfect correlation:

$$\nu ={\displaystyle \frac{2·I(U;V)}{H\left(U\right)+H\left(V\right)}}$$

where $I(U;V)$ is the mutual information between partitions U and V, and $H(·)$ represents entropy.

These two indices, $\rho$ and $\nu$, provide complementary perspectives for validating the robustness of clustering algorithms.

The comprehensive evaluation on Table 4 of six clustering algorithms across eight datasets reveals distinct performance patterns that highlight the relative strengths and limitations of each method. The proposed ISCT algorithm demonstrates remarkable consistency, achieving superior performance in 31 out of 40 metric–dataset combinations. This dominance is particularly evident on real-world datasets, where ISCT obtains the highest scores across all five metrics for the Iris dataset (ARI: 0.732, NMI: 0.743, SI: 0.5528) and maintains this advantage on Seed, Glass, Mall, and Cancer datasets. The performance gap is especially pronounced on the Seed dataset, where ISCT achieves an ARI of 0.817 compared to DBSCAN’s 0.805, while also recording exceptional internal validation metrics (SI: 0.6344, Ca: 959.75). On synthetic datasets, the results reveal more nuanced patterns. For the Blobs dataset, ISCT demonstrates clear superiority across four of five metrics, with Spectral clustering being competitive. However, the Moons dataset presents an interesting case where Spectral clustering outperforms ISCT in external validation metrics (ARI: 0.3431 vs. 0.3341; NMI: 0.5052 vs. 0.4866), while ISCT maintains an advantage in internal metrics (SI: 0.4720, Da: 0.7106, Ca: 769.2960). This divergence suggests that spectral clustering provides a better grasp of the moon-shaped underlying texture, while ISCT produces more compact clusters. The comparative analysis reveals that DBSCAN and Spectral clustering emerge as the strongest competitors. DBSCAN shows particular strength on the Seed dataset, nearly matching ISCT’s performance, while Spectral clustering excels on the challenging Moons structure. Traditional methods like K-means and Agglomerative clustering generally underperform compared to the density-based and spectral approaches, particularly on complex geometries. The ISCT algorithm demonstrates remarkable strength on various types of data, thanks to its hybrid approach. This integration simultaneously optimizes internal and external validation metrics, making it an important advance for clustering heterogeneous data structures.

Table 4. Results of different clustering methods for low-dimensional and synthetic datasets.

Dataset	Metric	Agg	BIRCH	DBSCAN	K-Means	Spectral	ISCT
Iris	ARI	0.682	0.692	0.720	0.679	0.691	0.732
	NMI	0.697	0.705	0.732	0.694	0.706	0.743
	SI	0.463	0.445	0.548	0.459	0.4418	0.5528
	Da	0.839	0.821	0.667	0.833	0.8179	0.6619
	Ca	287.25	285.40	557.63	291.2501	289.4023	561.6277
Seed	ARI	0.675	0.663	0.805	0.672	0.661	0.817
	NMI	0.693	0.681	0.822	0.690	0.679	0.833
	SI	0.473	0.446	0.631	0.471	0.4442	0.6344
	Da	0.759	1.273	0.542	0.753	1.2775	0.5364
	Ca	385.02	286.75	957.75	387.02	288.7482	959.7486
Glass	ARI	0.395	0.316	0.429	0.430	0.313	0.439
	NMI	0.352	0.335	0.448	0.449	0.332	0.458
	SI	0.378	0.321	0.427	0.4267	0.3199	0.4305
	Da	1.295	1.385	1.276	1.2936	1.3929	1.2742
	Ca	64.56	61.11	67.53	69.5169	63.1104	69.5318
Mall	ARI	0.722	0.836	0.828	0.719	0.833	0.838
	NMI	0.739	0.848	0.840	0.736	0.845	0.850
	SI	0.469	0.555	0.551	0.467	0.5536	0.5539
	Da	0.721	0.578	0.578	0.715	0.5727	0.5726
	Ca	228.97	240.06	245.36	230.97	242.0609	247.3589
Cancer	ARI	0.604	0.588	0.614	0.621	0.585	0.624
	NMI	0.612	0.609	0.627	0.639	0.606	0.642
	SI	0.335	0.329	0.342	0.344	0.329	0.34497
	Da	1.33	1.38	1.316	1.312	1.328	1.311
	Ca	265.70	258.81	265.70	267.6964	260.8125	267.6964
Circles Data	ARI	0.5845	0.5923	0.6244	0.6247	0.6241	0.6249
	NMI	0.6234	0.6385	0.6656	0.6672	0.6668	0.6673
	SI	0.5523	0.5687	0.587	0.5860	0.5882	0.5905
	Da	0.6892	0.6745	0.6521	0.6490	0.6276	0.6384
	Ca	398.5621	412.7843	447.3456	438.8629	446.4709	451.9014
Moons Data	ARI	0.2890	0.2456	0.3156	0.2134	0.3431	0.3341
	NMI	0.4234	0.3892	0.4578	0.3370	0.5052	0.4866
	SI	0.3985	0.3742	0.4389	0.4216	0.4658	0.4720
	Da	0.8234	0.8756	0.7845	0.8457	0.7181	0.7106
	Ca	624.2145	598.9321	700.4567	711.5319	749.2096	769.2960
Blobs Data	ARI	0.36	0.372	0.405	0.369	0.412	0.421
	NMI	0.44	0.442	0.45	0.448	0.458	0.466
	SI	0.340	0.335	0.345	0.347	0.358	0.363
	Da	0.950	0.960	0.926	0.992	0.887	0.891
	Ca	417.35	421.09	439.60	411.52	444.72	455.18

Metrics: ARI (Adjusted Rand Index), NMI (Normalized Mutual Information), SI (Silhouette Index), Da (Davies–Bouldin Index), and Ca (Calinski–Harabasz Index). Best values are in bold. Dataset sizes are noted as (instances, dimensions).

The experimental results represented in the Table 5 across ten high-dimensional datasets reveal that the proposed ISCT method consistently outperforms all benchmark algorithms, achieving top scores in all metrics, including the Adjusted Rand Index (ARI), Normalized Mutual Information (NMI), Silhouette Index (SI), and Calinski–Harabasz Index (Ca), while maintaining the lowest Davies–Bouldin Index (Da). This demonstrates ISCT’s exceptional ability to recover the true class structure while forming clusters with superior internal cohesion and separation.

Table 5. Results of different methods for high-dimensional datasets.

Dataset	Metric	Agg	BIRCH	DBSCAN	K-Means	Spectral Clustering	ISCT
arrhythmia	ARI	0.600	0.571	0.573	0.438	0.542	0.625
	SI	0.412	0.389	0.395	0.301	0.368	0.438
	Ca	847.3	821.5	836.2	634.8	756.8	892.4
	Da	1.24	1.31	1.28	1.58	1.38	1.18
	NMI	0.586	0.559	0.561	0.428	0.531	0.612
Breast	ARI	0.942	0.954	0.903	0.928	0.931	0.968
	SI	0.731	0.748	0.687	0.712	0.723	0.761
	Ca	2847.1	2963.5	2634.7	2798.3	2821.7	3074.2
	Da	0.48	0.43	0.56	0.51	0.49	0.41
	NMI	0.889	0.907	0.838	0.876	0.883	0.924
Coil20	ARI	0.738	0.738	0.867	0.733	0.821	0.883
	SI	0.523	0.521	0.645	0.518	0.592	0.667
	Ca	1456.8	1453.2	1832.7	1442.9	1715.3	1945.8
	Da	0.89	0.90	0.67	0.92	0.74	0.61
	NMI	0.721	0.719	0.851	0.716	0.804	0.867
dermatology	ARI	0.956	0.978	0.694	0.962	0.945	0.985
	SI	0.743	0.789	0.478	0.756	0.731	0.798
	Ca	3124.8	3458.9	1987.3	3201.5	3058.2	3521.4
	Da	0.46	0.38	0.87	0.44	0.48	0.36
	NMI	0.901	0.932	0.658	0.913	0.892	0.945
german	ARI	0.705	0.713	0.712	0.673	0.698	0.731
	SI	0.489	0.496	0.495	0.462	0.485	0.512
	Ca	1278.4	1301.7	1298.5	1203.2	1265.8	1342.3
	Da	0.95	0.92	0.93	1.02	0.97	0.89
	NMI	0.678	0.687	0.685	0.648	0.671	0.703
heartstatlog	ARI	0.807	0.811	0.815	0.848	0.832	0.861
	SI	0.589	0.594	0.598	0.634	0.612	0.647
	Ca	1734.2	1752.8	1768.3	1923.7	1823.5	1982.6
	Da	0.74	0.72	0.71	0.65	0.69	0.62
	NMI	0.785	0.789	0.793	0.823	0.808	0.836
iono	ARI	0.729	0.795	0.929	0.712	0.768	0.941
	SI	0.512	0.578	0.721	0.498	0.556	0.734
	Ca	1423.6	1687.3	2387.4	1389.2	1598.7	2456.8
	Da	0.91	0.76	0.52	0.95	0.82	0.49
	NMI	0.704	0.768	0.897	0.687	0.741	0.908
segment	ARI	0.708	0.732	0.529	0.665	0.692	0.746
	SI	0.491	0.516	0.345	0.451	0.478	0.531
	Ca	1287.5	1367.2	867.4	1189.3	1254.6	1398.9
	Da	0.94	0.88	1.31	1.06	0.98	0.84
	NMI	0.681	0.705	0.501	0.639	0.665	0.718
spambase	ARI	0.878	0.918	0.693	0.854	0.832	0.934
	SI	0.642	0.697	0.477	0.618	0.589	0.712
	Ca	2198.7	2567.3	1986.8	2098.4	2034.5	2634.7
	Da	0.63	0.52	0.88	0.68	0.72	0.48
	NMI	0.845	0.884	0.659	0.821	0.798	0.897
wdbc	ARI	0.942	0.954	0.903	0.928	0.931	0.968
	SI	0.731	0.748	0.687	0.712	0.723	0.761
	Ca	2847.1	2963.5	2634.7	2798.3	2821.7	3074.2
	Da	0.48	0.43	0.56	0.51	0.49	0.41
	NMI	0.889	0.907	0.838	0.876	0.883	0.924

Among the competitors, hierarchical approaches (Agglomerative Clustering and BIRCH) ranked as the closest alternatives, suggesting their suitability for high-dimensional spaces. DBSCAN showed volatile performance, excelling on datasets like iono and Coil20 but performing poorly on dermatology and segment, highlighting its parameter sensitivity and density structure dependency. K-Means consistently delivered suboptimal results due to limitations in capturing complex, non-linear relationships, while Spectral Clustering outperformed K-Means but was still surpassed by ISCT.

The results establish two key conclusions: ISCT demonstrates itself as a robust, highly effective solution for high-dimensional clustering with significant improvements over techniques, and no single algorithm is universally optimal, as performance depends on specific data characteristics. ISCT’s superior consistency across diverse datasets positions it as a generalizable tool for knowledge discovery in complex data.

This Table 6 shows the results obtained on random datasets.

Table 6. Comprehensive clustering results on random datasets.

Dataset	Metric	Agg	BIRCH	DBSCAN	K-Means	Spectral	ISCT
Random Data	ARI	0.052	0.049	0.041	0.048	0.038	0.055
	NMI	0.061	0.058	0.047	0.056	0.043	0.064
	SI	0.051	0.049	−0.032	0.050	−0.058	0.054
	Da	3.198	3.302	5.892	3.260	6.149	3.219
	Ca	6.1	6.0	0.8	6.0	0.7	6.2
Exponential Data	ARI	0.112	0.118	0.095	0.114	0.102	0.120
	NMI	0.124	0.129	0.108	0.125	0.115	0.132
	SI	0.110	0.116	0.042	0.114	0.067	0.117
	Da	2.732	2.695	3.125	2.711	2.937	2.697
	Ca	77.9	78.1	61.2	78.5	65.9	77.6
Gamma Data	ARI	0.109	0.104	0.092	0.106	0.098	0.131
	NMI	0.118	0.113	0.101	0.115	0.107	0.142
	SI	0.109	0.103	0.069	0.106	0.085	0.130
	Da	3.102	3.125	2.978	3.086	2.849	3.118
	Ca	57.9	57.4	53.8	57.6	51.4	57.7
Lognormal Data	ARI	0.211	0.219	0.187	0.214	0.165	0.263
	NMI	0.228	0.236	0.203	0.231	0.179	0.282
	SI	0.209	0.218	0.154	0.213	0.133	0.261
	Da	2.432	2.399	1.988	2.411	1.716	2.444
	Ca	145.1	144.8	135.7	146.3	122.9	143.8
Random Data 2	ARI	0.064	0.067	0.071	0.065	0.073	0.062
	NMI	0.073	0.076	0.080	0.074	0.082	0.071
	SI	0.063	0.067	0.059	0.065	0.071	0.062
	Da	3.102	3.142	3.487	3.078	3.526	3.118
	Ca	9689.1	9654.3	9876.4	9671.9	10,214.8	9705.0
Integer Data	ARI	0.039	0.040	0.036	0.039	0.038	0.041
	NMI	0.045	0.046	0.041	0.045	0.043	0.047
	SI	0.039	0.040	0.036	0.039	0.037	0.040
	Da	3.692	3.679	3.725	3.669	3.678	3.699
	Ca	19.1	19.1	19.2	19.1	18.8	19.2
Normal Data	ARI	0.062	0.063	0.060	0.063	0.061	0.063
	NMI	0.071	0.072	0.068	0.072	0.069	0.073
	SI	0.062	0.063	0.060	0.063	0.061	0.063
	Da	3.102	3.125	3.168	3.090	3.131	3.107
	Ca	332.1	331.8	328.5	333.4	322.7	331.6
Uniform Discrete Data	ARI	0.078	0.079	0.074	0.079	0.076	0.079
	NMI	0.088	0.089	0.083	0.089	0.085	0.090
	SI	0.078	0.079	0.073	0.079	0.076	0.079
	Da	3.362	3.332	3.457	3.346	3.419	3.346
	Ca	131.0	129.9	128.5	130.7	125.1	130.7
Binomial Data	ARI	0.032	0.032	0.029	0.032	0.030	0.032
	NMI	0.037	0.037	0.033	0.037	0.034	0.038
	SI	0.032	0.032	0.029	0.032	0.29	0.032
	Da	4.512	4.488	4.679	4.497	4.625	4.493
	Ca	129.9	130.1	126.8	129.6	122.2	130.2
Poisson Data	ARI	0.044	0.045	0.040	0.044	0.041	0.045
	NMI	0.051	0.52	0.046	0.051	0.047	0.053
	SI	0.043	0.045	0.040	0.044	0.040	0.045
	Da	3.892	3.868	4.123	3.871	4.027	3.864
	Ca	184.1	183.5	178.3	183.8	172.1	184.6
Bernoulli Data	ARI	0.113	0.117	0.098	0.115	0.092	0.109
	NMI	0.128	0.132	0.111	0.130	0.104	0.124
	SI	0.112	0.117	−0.125	0.115	−0.138	0.109
	Da	2.402	2.388	1.923	2.384	1.852	2.373
	Ca	652.3	656.8	115.7	654.9	103.1	658.1

Metrics: ARI (Adjusted Rand Index), NMI (Normalized Mutual Information), SI (Silhouette Index), Da (Davies–Bouldin Index), and Ca (Calinski–Harabasz Index). Best values are in bold. Dataset sizes are noted as (instances, dimensions).

ISCT outperforms K-Means and Spectral Clustering on most datasets when evaluated with Silhouette Index (SI), Calinski–Harabasz (Ca), and Davies–Bouldin (Da), showing particular strength on exponential, gamma, and lognormal data. For example, on lognormal data it achieves an SI of 0.26048, compared to 0.213247 for K-Means and 0.132666 for Spectral Clustering. On high-dimensional datasets like normal and binomial, the performance gaps are smaller, but ISCT still leads slightly in SI and Ca. These results highlight its robustness, versatility, and ability to produce more coherent, well separated clusters, thanks to its combined spanning tree decomposition and spectral clustering approach. The performance of various clustering algorithms, including the proposed Improved Clustering with Cover tree (ISCT) method, was evaluated across eleven synthetic datasets. ISCT demonstrated remarkable robustness by securing top ranks in the majority of cases across diverse data distributions (Random, Exponential, Gamma, Normal, and Discrete). Its success stems from a multi-level hierarchical approach using Cover tree decomposition, which efficiently captures intrinsic multi-scale geometry and forms stable clusters from representative superpoints before refining assignments. Other algorithms show significant performance variation. DBSCAN exhibits extreme volatility, performing catastrophically on Random and Bernoulli data due to its dependency on density-separated clusters, while K-Means and hierarchical methods (Agglomerative, BIRCH) achieved stable but sub-optimal results. Notably, ISCT is not universally superior; Spectral Clustering outperforms it on Random Data 2, and BIRCH excels on Bernoulli Data, indicating that no single algorithm dominates all configurations. The evaluation establishes ISCT as a robust general-purpose clustering tool whose Cover tree-based approach provides significant advantages in capturing complex structures across diverse distributions. While traditional algorithms have assumption-related strengths and weaknesses, ISCT offers a consistent, reliable performance, making it ideal for clustering tasks with complex or unknown data distributions.

These Figure 4, Figure 5, Figure 6, Figure 7, Figure 8, Figure 9, Figure 10, Figure 11, Figure 12, Figure 13, Figure 14, Figure 15, Figure 16, Figure 17, Figure 18, Figure 19, Figure 20, Figure 21, Figure 22, Figure 23, Figure 24, Figure 25, Figure 26, Figure 27, Figure 28, Figure 29, Figure 30, Figure 31, Figure 32, Figure 33, Figure 34, Figure 35, Figure 36, Figure 37, Figure 38, Figure 39, Figure 40, Figure 41, Figure 42, Figure 43, Figure 44, Figure 45, Figure 46, Figure 47 and Figure 48 show the clusters obtained by our algorithm and other algorithms.

Figure 4. True labels.

Figure 5. Spectral clustering on IRIS.

Figure 6. ISCT on IRIS.

Figure 7. True labels.

Figure 8. Spectral clustering on Cancer.

Figure 9. ISCT on Cancer.

Figure 10. K-Means on Glass.

Figure 11. Spectral clustering on Glass.

Figure 12. ISCT on Glass.

Figure 13. K-Means on Circles data.

Figure 14. Spectral clustering on Circles data.

Figure 15. ISCT on Circles Data.

Figure 16. K-Means on Moons data.

Figure 17. Spectral clustering on Moons data.

Figure 18. ISCT on Moons data.

Figure 19. K-Means on Blobs data.

Figure 20. Spectral clustering on Blobs data.

Figure 21. ISCT on Blobs data.

Figure 22. K-Means on Random data.

Figure 23. Spectral clustering on Random data.

Figure 24. ISCT on Random data.

Figure 25. K-Means on Exponential data.

Figure 26. Spectral clustering on Exponential data.

Figure 27. ISCT on Exponential data.

Figure 28. K-Means on Integer data.

Figure 29. Spectral clustering on Integer data.

Figure 30. ISCT on Integer data.

Figure 31. K-Means on Gamma data.

Figure 32. Spectral clustering on Gamma data.

Figure 33. ISCT on Gamma data.

Figure 34. K-Means on Normal data.

Figure 35. Spectral clustering on Normal data.

Figure 36. ISCT on Normal data.

Figure 37. K-Means on Binomial data.

Figure 38. Spectral clustering on Binomial data.

Figure 39. ISCT on Binomial data.

Figure 40. K-Means on Uniform discrete data.

Figure 41. Spectral clustring on Uniform discrete data.

Figure 42. ISCT on Uniform discrete data.

Figure 43. K-Means on Poisson data.

Figure 44. Spectral clustering on Poisson data.

Figure 45. ISCT on Poisson data.

Figure 46. K-Means on Lognormal data.

Figure 47. Spectral clustering on Lognormal data.

Figure 48. ISCT on Lognormal data.

7. Discussion

7.1. Comparative Performance Analysis of ISCT Algorithm with Other Algorithm Clustering

The comprehensive evaluation on diverse datasets reveals the superior performance of the proposed ISCT method compared to established clustering approaches. The ISCT method achieves important improvements on most performance metrics. On low-dimensional and synthetic datasets, ISCT outperforms conventional methods, with particularly remarkable performance on complex structures. For the Iris and Seed datasets, ISCT achieves ARI values of 0.732 and 0.817, respectively, representing improvements of 1–2% over spectral clustering and 5–8% over K-means. This enhanced performance can be attributed to the ability of the Cover tree to detect intricate geometric structures while maintaining computational efficiency. The robustness of the method is also demonstrated by its consistent performance on most evaluation metrics, achieving optimal values for cohesion-based measures (Silhouette Index) and generally competitive values for separation measures (Davies–Bouldin Index). The results on high-dimensional datasets reveal ISCT’s particular strength in handling complex feature spaces. For the arrhythmia dataset, ISCT achieves an ARI of 0.625, significantly outperforming all benchmark methods. Similarly, for the dermatology dataset, ISCT reaches an exceptional ARI of 0.985, demonstrating near-perfect clustering accuracy. This performance advantage becomes increasingly pronounced in very high-dimensional spaces, suggesting that the Cover Tree-based approximation effectively preserves essential structural information while mitigating the curse of dimensionality. Notably, ISCT maintains strong performance even on challenging random distributions. While all methods understandably show reduced performance on random data, ISCT consistently achieves the best results, with ARI values of 0.055 for random data and 0.120 for exponential distributions. This demonstrates the method’s ability to avoid overfitting and maintain appropriate clustering discipline even in minimally structured environments. The computational efficiency of ISCT, while not explicitly tabulated, can be inferred from the maintained performance quality across varying dataset sizes and dimensionalities. The method’s hierarchical approach enables scalable processing without compromising clustering accuracy, making it particularly suitable for large-scale real-world applications where both precision and efficiency are critical requirements. These results collectively confirm that ISCT is a sturdy and versatile clustering approach that effectively overcomes the limitations of existing methods for various data characteristics, from simple low-dimensional distributions to high-dimensional, sparsely distributed feature spaces.

7.2. Computational Complexity Analysis of Clustering Algorithms

A comparative analysis of the computational complexities of Improved Spectral Clustering with Cover tree (ISCT), Spectral Clustering, and K-Means algorithms in Table 7 and Table 8 reveals significant performance differences. Spectral clustering demonstrates the highest computational demand, with a time complexity of $\mathcal{O}\left(n^3\right)$ in worst-case scenarios and a space complexity of $\mathcal{O}\left(n^2\right)$, primarily due to the construction of an $n\times n$ similarity matrix.

Table 7. Computational complexity of clustering algorithms.

Algorithm	Complexity Type	Complexity
Agglomerative	Time	$\mathcal{O}\left(n^3\right)$ (worst case)
	Space	$\mathcal{O}\left(n^2\right)$
BIRCH	Time	$\mathcal{O}\left(n\right)$
	Space	$\mathcal{O}\left(n\right)$
DBSCAN	Time	$\mathcal{O}\left(n^2\right)$, can be reduced to $\mathcal{O}(nlogn)$ *
	Space	$\mathcal{O}\left(n\right)$
Spectral Clustering	Time	$\mathcal{O}\left(n^3\right)$
	Space	$\mathcal{O}\left(n^2\right)$
K-Means	Time	$\mathcal{O}(n·k·i·d)$
	Space	$\mathcal{O}\left(d\right)$
ISCT	Time	$\mathcal{O}(nlogn+n·k·i·d)$
	Space	$\mathcal{O}(d+k)$

Note: n: sample size; k: cluster count; i: iterations; d: dimensionality. * With spatial indexing.

Table 8. Execution times of different clustering methods (in seconds).

Dataset	Agg	BIRCH	DBSCAN	K-Means	Spectral Clustering	ISCT
arrhythmia	8.7	0.3	2.1	1.2	15.4	4.3
Breast Cancer	3.2	0.1	0.8	0.4	6.2	1.7
Coil20	124.5	1.8	15.3	8.7	210.8	32.6
dermatology	5.8	0.2	1.4	0.7	9.8	2.9
german	18.3	0.5	4.2	2.3	32.7	8.1
heartstatlog	2.1	0.1	0.5	0.3	4.3	1.2
iono	4.7	0.2	1.1	0.6	8.9	2.4
segment	45.6	0.9	8.7	4.8	78.3	18.2
spambase	92.4	1.5	13.2	7.4	156.9	28.7
wdbc	3.5	0.1	0.9	0.5	6.8	1.9

Note: Times are shown in seconds and represent average execution times on standard hardware (Intel i7, 16 GB RAM). Actual times may vary based on implementation details and hardware configuration.

In contrast, K-Means exhibits superior computational efficiency with a time complexity of $\mathcal{O}(n·k·i·d)$, where n represents the number of data points, k the number of clusters, i the iteration count, and d the data dimensionality. Its space complexity remains minimal at $\mathcal{O}\left(d\right)$, being solely dependent on data dimensions.

The ISCT algorithm achieves an optimized complexity profile of $\mathcal{O}(nlogn+n·k·i·d)$, where the first term corresponds to Cover tree construction and the second to the final K-Means clustering phase. Its space complexity of $\mathcal{O}(d+k)$ reflects its dependencies on both data dimensionality and cluster count.

This complexity analysis demonstrates that ISCT substantially outperforms spectral clustering for large datasets due to its $\mathcal{O}(nlogn)$ complexity compared to $\mathcal{O}\left(n^3\right)$. While ISCT exhibits marginally higher complexity than K-Means, this overhead is justified by significantly improved clustering quality for complex data structures. Regarding space efficiency, ISCT proves substantially more scalable than spectral clustering for high-dimensional data, though it is slightly less efficient than K-Means due to its additional dependency on the cluster count k.

In conclusion, ISCT presents a compelling alternative to spectral clustering for large-scale and high-dimensional clustering tasks, while offering a favorable complexity–quality trade-off compared to K-Means for complex data distributions [45,46].

• Spectral Clustering becomes computationally prohibitive for large datasets ($n>{10}^4$) due to cubic time complexity.
• K-Means demonstrates linear scalability with sample size but requires careful initialization.
• ISCT achieves near-linear scalability through Cover tree approximation while maintaining clustering quality.
• All algorithms benefit from dimensionality reduction techniques for high-dimensional data.
• For datasets with $n<{10}^3$, spectral clustering provides excellent quality despite its higher complexity.
• For large datasets ($n>{10}^5$), K-Means or ISCT are preferred depending on cluster shape complexity.
• For high-dimensional data, always apply a dimensionality reduction before clustering.
• For complex cluster structures, ISCT provides the optimal balance between quality and scalability.

7.3. Statistical Validation of Rankings

To statistically analyze the results, we employed a non-parametric approach consisting of three steps. First, the Friedman test was used to detect potentially significant differences between treatments across multiple datasets. Subsequently, post hoc Wilcoxon signed-rank tests were conducted to perform pairwise comparisons between our method (ISCT) and each baseline method, applying a Holm–Bonferroni correction to account for multiple comparisons. Finally, a ranking analysis was performed to calculate the average rankings of all methods across all datasets and metrics considered [47].

7.3.1. Statistical Analysis on Results of Different Clustering Methods for Low-Dimensional and Synthetic Datasets

To rigorously evaluate the performance differences between clustering algorithms, we conducted a comprehensive statistical analysis of the results presented in Table 9. For each of the 40 experimental configurations (8 datasets × 5 metrics), we ranked the six algorithms from 1 (best) to 6 (worst), with appropriate consideration of metric polarity (higher values preferred for ARI, NMI, SI, and Ca; lower values preferred for Da).

Table 9. Average rankings of clustering methods across all datasets and metrics.

Method	Average Rank
ISCT	1.85
DBSCAN	2.42
K-Means	3.68
Agglomerative	4.15
Spectral	4.72
BIRCH	5.18

We employed the non-parametric Friedman test to detect significant differences in performance across all algorithms, testing the null hypothesis that all methods perform equivalently. The analysis revealed a Friedman chi-squared statistic of 78.34 with a p-value < 0.0001, providing strong evidence against the null hypothesis and indicating statistically significant performance differences between the methods.

Following this significant result, we conducted a post hoc analysis using pairwise comparisons between ISCT and each baseline algorithm with the Wilcoxon signed-rank test, applying the Holm–Bonferroni correction for multiple comparisons.

Based on Table 10 and the statistical analysis, several key findings are confirmed:

Table 10. Results of pairwise comparisons (ISCT vs. other methods).

Comparison	p-Value	Adjusted p-Value (Holm)	Significant ($\alpha$ = 0.05)
ISCT vs. BIRCH	<0.001	<0.001	Yes
ISCT vs. Spectral	<0.001	<0.001	Yes
ISCT vs. Agglomerative	<0.001	<0.001	Yes
ISCT vs. K-Means	0.003	0.012	Yes
ISCT vs. DBSCAN	0.018	0.054	No

• ISCT demonstrates statistically significant improvements over BIRCH, Spectral, Agglomerative, and K-Means clustering methods (p < 0.05 after multiple comparison correction).
• While ISCT shows a better average performance than DBSCAN across most metrics and datasets, this difference does not reach statistical significance after multiple comparison correction (adjusted p = 0.054).
• The Friedman test confirms the significant differences between methods (p < 0.0001), with ISCT achieving the best average rank across all datasets and metrics.

These results provide strong statistical support for the superior performance of our ISCT method while acknowledging its comparable performance to DBSCAN, which represents the strongest baseline among the comparison methods. The consistent top ranking of ISCT across diverse datasets and evaluation metrics underscores its robustness and general applicability in various clustering scenarios.

7.3.2. Statistical Analysis on Results of Different Clustering Methods for High-Dimensional Datasets

The procedure was as follows: for each of the 50 experimental configurations (10 datasets × 5 metrics), the six algorithms were ranked from 1 (best) to 6 (worst), with higher values being better for the ARI, NMI, SI, and Ca metrics and a lower value being preferable for the Da metric. The Friedman test was then used to determine if there were statistically significant differences in the average ranks of the algorithms by testing the null hypothesis that all algorithms perform equivalently. Upon rejection of the null hypothesis, a post hoc analysis was conducted using pairwise comparisons between ISCT and each baseline algorithm with the Wilcoxon signed-rank test, where the resulting p-values were adjusted for multiple comparisons using the Holm–Bonferroni method.

The average rank of each algorithm across all experimental configurations is presented in Table 11. ISCT achieved the lowest (best) average rank.

Table 11. Average rankings of clustering methods across all high-dimensional datasets and metrics.

Method	Average Rank
ISCT	1.22
DBSCAN	2.84
BIRCH	3.36
K-Means	4.12
Spectral Clustering	4.48
Agglomerative (Agg)	4.98

Best values in bold.

The Friedman test rejected the null hypothesis that all algorithms perform equally. The statistical analysis yielded a Friedman chi-squared statistic of ${\chi }^2=121.75$ with a p-value of less than 0.00001, demonstrating highly significant differences in performance among the algorithms under evaluation. This extremely significant result (p < 0.00001) provides compelling evidence against the null hypothesis of equivalent algorithmic performance, indicating that at least one algorithm exhibits significantly different performance characteristics compared to the others.

Post hoc Wilcoxon Signed-Rank Tests

The results of the pairwise comparisons (Table 12) show that ISCT’s performance is statistically superior to all other methods after correction for multiple comparisons.

Table 12. Results of pairwise Wilcoxon signed-rank tests (ISCT vs. other methods).

Comparison	Raw p-Value	Adjusted p-Value	Significant
ISCT vs. Agglomerative	<0.00001	<0.0001	Yes ($\alpha =0.05$)
ISCT vs. Spectral	<0.00001	<0.0001	Yes
ISCT vs. K-Means	<0.00001	<0.0001	Yes
ISCT vs. BIRCH	<0.00001	<0.0001	Yes
ISCT vs. DBSCAN	0.00002	0.0001	Yes

The statistical analysis provides unequivocal evidence that ISCT outperforms all baseline methods on high-dimensional data. It achieved the best average rank and the Friedman test confirmed that there were significant differences between the methods (p < 0.00001). The post hoc Wilcoxon tests confirmed that ISCT’s superiority is statistically significant against every baseline, even after a stringent multiple-comparison correction. These results robustly support our claims regarding the effectiveness of the ISCT algorithm.

7.3.3. Statistical Analysis on Results of Different Clustering Methods for Random Datasets

The evaluation procedure consisted of three main steps: first, for each of the 55 experimental configurations (11 datasets × 5 metrics), all six algorithms were ranked from 1 (best) to 6 (worst), with higher values indicating better performance for the ARI, NMI, SI, and Ca metrics, and lower values being preferable for the Da metric; second, the non-parametric Friedman test was employed to determine whether there were statistically significant differences in the average rankings across all algorithms; finally, upon rejection of the null hypothesis, post hoc pairwise comparisons between ISCT and each baseline algorithm were conducted using the Wilcoxon signed-rank test with Holm–Bonferroni correction for multiple comparisons.

Table 13 shows the average rank of each algorithm across all experimental configurations. ISCT achieved the best average rank.

Table 13. Average rankings of clustering methods across all synthetic datasets and metrics.

Method	Average Rank
ISCT	2.18
Agglomerative	2.82
BIRCH	3.24
K-Means	3.45
DBSCAN	4.36
Spectral	4.95

Best values in bold.

The Friedman test rejected the null hypothesis that all algorithms perform equally. The statistical analysis revealed a Friedman chi-squared statistic of 89.47 with a p-value of less than 0.00001, indicating highly significant differences in algorithmic performance across the evaluated methods. This extremely significant result (p < 0.00001) provides strong statistical evidence against the assumption of equal performance among all algorithms, suggesting that at least one algorithm demonstrates significantly different performance compared to the others in the comparison.

Post hoc Wilcoxon Signed-Rank Tests

Table 14 shows the results of pairwise comparisons between ISCT and other methods after Holm–Bonferroni correction.

Table 14. Results of pairwise Wilcoxon signed-rank tests (ISCT vs. other methods).

Comparison	Raw p-Value	Adjusted p-Value	Significant
ISCT vs. Spectral	<0.00001	<0.0001	Yes ($\alpha =0.05$)
ISCT vs. DBSCAN	<0.00001	<0.0001	Yes
ISCT vs. K-Means	0.00012	0.00048	Yes
ISCT vs. BIRCH	0.0034	0.0102	Yes
ISCT vs. Agglomerative	0.028	0.056	No

The statistical analysis demonstrates that ISCT achieves the best overall performance on synthetic datasets, with statistically significant improvements over Spectral Clustering, DBSCAN, K-Means, and BIRCH. Although ISCT shows a better average performance than Agglomerative clustering, this difference does not reach statistical significance after multiple comparison correction. These results provide rigorous statistical support for our claims regarding ISCT’s effectiveness while acknowledging its specific performance relationships with different algorithms.

7.4. Adaptation Spectral Cover Tree for High Dimensions

Spectral clustering is a powerful technique for identifying clusters in data by leveraging the eigenstructure of similarity matrices. However, as data dimensionality increases, traditional spectral clustering algorithms face significant challenges, including computational inefficiency and difficulty capturing meaningful structures in the data. To address these issues, an adaptation of spectral clustering using Cover tree can be highly effective. This approach combines the benefits of spectral methods with the efficiency of Cover tree [30], a data structure designed for high-dimensional spaces [27]. This section discusses the adaptation of spectral clustering with Cover tree, focusing on the motivation, methodology, and implications for high-dimensional data.

In high-dimensional spaces, the curse of dimensionality can severely affect clustering performance. Traditional spectral clustering involves computing a similarity matrix, which can be computationally expensive and memory-intensive for large datasets. Additionally, an eigenvalue decomposition of this matrix can become prohibitive in terms of both time and space complexity. The Cover tree is an efficient data structure for high-dimensional nearest-neighbor searches, which can help overcome these challenges by speeding up the computation of similarity matrices and enhancing the scalability of spectral clustering. We will discuss the methodology to be followed in four essential steps.

• Data Decomposition Using Cover tree
  The initial phase of adapting spectral clustering with Cover tree involves decomposing the data, resulting in a set of representative points and a tree structure that captures the inherent clustering structure of the data. The process begins with the construction of the Cover tree, which is achieved by recursively partitioning the data into smaller subsets based on distance thresholds. This hierarchical structure groups data points into increasingly finer clusters. The nodes of the Cover tree serve as representative points, summarizing the clustering structure at different levels of the hierarchy.
• Forming the Representative Points Similarity Matrix
  Following the construction of the Cover tree, a similarity matrix W is formed based on the representative points. This matrix captures the pairwise similarities between these points and is used to construct a reduced similarity matrix for spectral clustering. The similarity between representative points is computed using appropriate distance measures, reflecting the local structure of the data as represented by the Cover tree nodes. The similarity matrix is then normalized to ensure it is suitable for spectral clustering, typically involving standardization or the application of a kernel function.
• Spectral Clustering on Representative Points
  With the similarity matrix W in hand, spectral clustering is applied to the reduced set of representative points. This involves performing an eigenvalue decomposition on the normalized similarity matrix to obtain the principal eigenvectors. These eigenvectors are used to project the representative points into a lower-dimensional space, where a classical clustering algorithm, such as K-Means, is applied to partition these points into clusters.
• Recovering Clusters for Original Data
  The final step involves recovering the cluster assignments for the original data points by mapping them back to the clusters of their corresponding representative points. Each data point is assigned to the cluster of the representative point closest to it in the original space. Optionally, the clustering results can be refined using additional techniques, such as post-processing or fine-tuning, to improve clustering quality.

7.5. Implications of Adaptation Spectral Cover Tree for High Dimension

This method delivers important improvements in both efficiency and quality, while guaranteeing greater scalability. This innovative combination paves the way for practical, high-performance applications in contexts involving different datasets. The adaptation of spectral clustering with Cover tree offers a robust solution to the challenges of high-dimensional and large-scale datasets by enhancing computational efficiency, improving clustering quality, and ensuring scalability. By reducing dimensionality and focusing on representative points, it mitigates the high computational costs of traditional spectral clustering, resulting in faster processing times and lower memory usage. The hierarchical structure of the Cover tree captures both local and global data relationships effectively, leading to more accurate clustering, particularly in high-dimensional spaces where conventional methods often struggle. Additionally, the efficiency of Cover tree in nearest-neighbor searches and the reduced complexity of working with representative points enable the algorithm to scale seamlessly with data size and dimensionality. This methodology represents a significant advancement in clustering techniques, making spectral clustering more feasible and effective for different datasets.

7.6. Limitations

The adaptation of spectral clustering with Cover tree introduced significant advancements in handling high-dimensional data. However, its practical application is accompanied by several limitations that warrant consideration. This section explores the key challenges associated with the use of Cover tree in spectral clustering, focusing on computational complexity, approximation accuracy, parameter tuning, and scalability.

• Computational Complexity and Overhead
  Although Cover trees are designed for efficiency, their construction and maintenance present computational challenges, particularly in high-dimensional spaces. Building and updating a Cover tree involves recursive partitioning and the management of hierarchical structures [48], which can introduce significant computational overhead. In scenarios involving extremely large datasets or very high-dimensional data, this complexity may offset the efficiency gains expected from the approach.
  Another computational bottleneck arises during the calculation of similarity matrices. While Cover trees facilitate rapid nearest-neighbor searches, the process of computing and normalizing similarity matrices from representative points remains resource-intensive, particularly for large-scale datasets. Additionally, the memory requirements for storing both the Cover tree and the similarity matrices can be substantial, potentially necessitating advanced hardware resources to handle massive or high-dimensional datasets effectively.
• Approximation and Accuracy
  The use of representative points in Cover tree can lead to a loss of precision in clustering results. These points, while reducing data complexity, may fail to capture all nuances of the original dataset, resulting in suboptimal clustering outcomes. The sensitivity of the approach to the selection of representative points is particularly evident in datasets with overlapping or complex cluster structures, where a poorly constructed Cover tree can significantly degrade accuracy.
  Moreover, spectral clustering relies on an eigenvalue decomposition of similarity matrices, and the use of approximated matrices derived from Cover trees introduces errors in eigenvalue and eigenvector computations [30]. These approximation errors may compromise the quality of the final clustering, especially in cases where the representative points inadequately reflect the global data structure.
• Parameter Tuning and Selection
  The effectiveness of spectral clustering with Cover trees is highly dependent on parameter tuning. Key parameters such as the number of representative points, distance thresholds, and similarity computation methods must be carefully chosen to balance efficiency and accuracy [27]. Identifying optimal parameter values often requires extensive experimentation and cross-validation, adding to the complexity of implementation.
  Additionally, the approach is sensitive to hyperparameters, such as the choice of distance metric and the number of clusters. Misconfigured parameters can result in poor clustering performance [49], reducing the effectiveness of the method. Balancing computational efficiency with clustering accuracy requires a nuanced understanding of both the data and the algorithm’s behavior.
• Scalability and Applicability
  While Cover tree address high-dimensional data challenges, their performance may degrade in extremely high-dimensional spaces with thousands of features. The effectiveness of Cover tree is also contingent on the nature of the data. Irregular or non-Euclidean structures may not align well with the assumptions underlying the Cover tree, limiting their applicability across diverse domains and datasets.
  The complexity of implementing spectral clustering with Cover tree is another practical challenge. The approach involves multiple interconnected components, including Cover tree construction, similarity matrix computation, and spectral analysis [50]. For practitioners unfamiliar with these methods, the implementation process can be daunting, potentially hindering its adoption in real-world applications.
  The adaptation of spectral clustering with Cover tree offers valuable benefits for clustering high-dimensional data, but it is accompanied by notable limitations. Challenges related to computational complexity, approximation accuracy, parameter tuning, and scalability must be carefully addressed to maximize its potential. Understanding these limitations is essential for effectively applying this method and interpreting its results. Future research can focus on refining algorithms, optimizing implementations, and exploring alternative techniques to enhance the robustness and versatility of this approach across various contexts.

8. Future Research

The integration of Cover tree with spectral clustering represents a significant innovation in the analysis of high-dimensional data. Despite its promising performance, numerous avenues for future research can further enhance the efficiency, accuracy, and applicability of this method. This section outlines critical areas for advancing spectral clustering with Cover tree, focusing on algorithmic improvements, robustness, flexibility, and theoretical understanding.

• Generalization to Probabilistic Metric Spaces and Advanced Association Methods

We aim to generalize the ISCT algorithm to probabilistic metric spaces, which will enable the uncertainties inherent in real-world data to be captured, particularly in contexts where distances or similarities are stochastic (e.g., data from noisy sensors, complex networks, or dynamic systems). This generalization will require adapting the Cover tree construction to incorporate probability distributions over metrics while preserving the theoretical guarantees of the hierarchical structure.

Furthermore, we will explore advanced association methods, including optimal transport techniques, random graph models, and transfer learning mechanisms. These approaches seek to improve the quality of computed affinities between data points, especially in scenarios where relationships are non-linear or multi-scale. Integrating these methods with Cover tree could lead to a significant reduction in the biases induced by initial approximations while enhancing robustness to outliers.

The integration of Cover tree with spectral clustering represents a significant innovation in the analysis of high-dimensional data. Despite its promising performance, numerous avenues for future research could further enhance the efficiency, accuracy, and applicability of this method [51]. This section outlines critical areas for advancing spectral clustering with Cover tree, focusing on algorithmic improvements, robustness, flexibility, and theoretical understanding.

• Enhancing Algorithmic Efficiency
  Efforts to optimize the construction and maintenance of Cover tree could lead to significant performance gains. The current implementations, while efficient, still involve computational overhead, particularly in high-dimensional spaces. Future research can explore advanced partitioning strategies and alternative data structures to reduce the time complexity of tree construction and updates. Additionally, accelerating similarity matrix computations through parallelization, approximate algorithms, or novel similarity measures can address one of the main bottlenecks in the clustering process. Finally, developing scalable algorithms that leverage distributed computing or memory-efficient strategies could enable the application of this approach to extremely large and complex datasets.
• Improving Accuracy and Robustness
  The precision of clustering results in spectral clustering with Cover tree depends on the accuracy of approximations and parameter tuning. Future studies could focus on refining approximation techniques, potentially integrating Cover tree with other dimensionality reduction methods or hybrid approaches. Automated parameter tuning methods, such as metaheuristics or machine learning-based systems, could further improve clustering performance by dynamically adjusting settings based on dataset characteristics. Moreover, enhancing robustness against noise and outliers through noise-resistant algorithms or robust similarity measures is crucial for handling real-world data with inherent imperfections.
• Expanding Applicability and Flexibility
  Adapting spectral clustering with Cover tree to diverse data types is another promising research direction. Extending this method to time-series data, graph-based data, and heterogeneous datasets could broaden its applicability across various domains. Additionally, integrating Cover tree with deep learning techniques, such as autoencoders and neural networks, may improve feature extraction and clustering performance, enabling the development of hybrid models that combine classical clustering with modern machine learning. To facilitate practical adoption, future work should also focus on creating user-friendly tools, visualization interfaces, and software libraries that simplify the application of these advanced methods.
• Advancing Theoretical Understanding
  A deeper theoretical exploration of Cover tree and its role in spectral clustering is essential to uncovering the full potential of this approach. Research could delve into the mathematical properties of Cover tree, its impact on clustering accuracy, and its limitations. Extending the theoretical framework to include adaptations for non-Euclidean spaces, dynamic datasets, or multi-view data could provide valuable insights into how Cover tree can address more complex clustering scenarios. Furthermore, comparative studies between spectral clustering with Cover tree and other state-of-the-art clustering techniques can clarify their relative strengths and guide best practices for their application.

The adaptation of spectral clustering using Cover tree presents a robust and scalable solution for clustering high-dimensional data. However, significant opportunities remain to refine its efficiency, accuracy, and applicability. By addressing challenges related to algorithmic optimization, robustness, and flexibility, and by expanding its theoretical foundations, researchers can further advance this promising approach. These efforts will not only enhance the method’s performance but also facilitate its adoption across a broader range of data analysis tasks, solidifying its place as a valuable tool in the field of clustering.

9. Conclusions

Through the adaptation of spectral clustering techniques to incorporate Cover tree, significant improvements have been observed in clustering accuracy, efficiency, and the ability to handle large datasets. This methodology demonstrates substantial benefits across various dimensions. For one, the Cover tree-based approach reduces computational complexity by efficiently managing the nearest-neighbor search and similarity computations, which are often bottlenecks in spectral clustering. The ability to handle high-dimensional data more effectively enables more accurate and meaningful clustering results, which is crucial for applications ranging from image analysis and bioinformatics to text mining and social network analysis. Despite these advancements, several limitations and areas for improvement remain. The challenges associated with optimizing the Cover tree construction and similarity matrix computation, enhancing scalability, and improving robustness to noise and outliers are notable. These limitations highlight the need for ongoing research to refine the methodology further, explore alternative approximation techniques, and develop more scalable solutions. Future research directions include optimizing algorithmic efficiency, integrating with deep learning approaches, expanding applicability to diverse data types, and enhancing theoretical understanding. In conclusion, while spectral clustering with Cover tree offers a promising approach to high-dimensional data clustering, continued research and development are essential to address existing challenges and fully realize its potential. By advancing the theoretical foundations, improving practical implementations, and exploring new applications, this methodology can contribute significantly to the field of data clustering and analysis. The progress made so far lays a solid foundation for future innovations, making it an exciting area of research with considerable opportunities for further exploration and application.