Ensemble Clustering Method via Robust Consensus Learning

Abstract

Although ensemble clustering methods based on the co-association (CA) matrix have achieved considerable success, they still face the following challenges: (1) in the label space, the noise within the connective matrices and the structural differences between them are often neglected, and (2) the rich structural information inherent in the feature space is overlooked. Specifically, for each connective matrix, a symmetric error matrix is first introduced in the label space to characterize the noise. Then, a set of mapping models is designed, each of which processes a denoised connective matrix to recover a reliable consensus matrix. Moreover, multi-order graph structures are introduced into the feature space to enhance the expressiveness of the consensus matrix further. To preserve a clear cluster structure, a theoretical rank constraint with a block-diagonal enhancement property is imposed on the consensus matrix. Finally, spectral clustering is applied to the refined consensus matrix to obtain the final clustering result. Experimental results demonstrate that ECM-RCL achieves superior clustering performance compared to several state-of-the-art methods.

1. Introduction

Clustering analysis [1,2] is a fundamental task in unsupervised learning. To further improve the robustness and reliability of clustering results, ensemble learning has been incorporated into clustering research [3,4,5,6,7,8,9]. Among ensemble clustering methods, methods based on the co-association (CA) matrix have been widely adopted, owing to their simplicity and strong theoretical foundation [10]. Typically, these methods follow a standard two-stage framework. In the first stage, multiple connective matrices are integrated to construct the CA matrix. Then, the CA matrix is used as input to a traditional clustering method (e.g., K-means [11]) to obtain the final clustering result. Due to the critical role of the CA matrix in ensemble clustering, a series of ensemble clustering methods [3,7,10,12,13,14,15,16,17,18,19,20] have been proposed to enhance the CA matrix in the following aspects:

First, some studies [3,12,16,19,20] enhance the CA matrix by introducing the weighting mechanism. For example, Bagherinia et al. [12] selected base clustering results based on their diversity and quality to construct a fuzzy CA matrix. Huang et al. [3] applied entropy theory to assign weights to clusters, thereby constructing a locally weighted CA matrix. Gu et al. [16] jointly considered uncertainties in base clustering results and clusters, using ensemble-driven cluster-uncertainty estimation to construct a two-level weighted CA matrix. Zheng et al. [20] enhanced the CA matrix by weighting base clustering results according to their correlations. Yang et al. [19] proposed a novel CA matrix learning method by dynamically adjusting cluster weights. Zeng et al. [21] designed a cluster confidence measure to assess low-dimensional deep embeddings, ensuring the generation of high-quality and diverse base clustering results for learning the CA matrix. Huang et al. [22] generated base clustering results by leveraging multi-level features from a neural network. An entropy criterion was then introduced to weight clusters, thereby providing a reliable basis for constructing the CA matrix.

Second, some methods [7,10,13,17,18] aim to enhance the structural representation of the CA matrix. For example, Tao et al. [13] improved the expressiveness of the CA matrix by leveraging its low-rank representation. Jia et al. [7] proposed a self-enhancement mechanism to refine the CA matrix representation. Xu et al. [17] employed a self-expressive model combined with self-enhancement to capture both global and local CA matrix structures, thereby improving clustering performance. Xu et al. [18] integrated a self-enhancement mechanism with topological consistency propagation to improve the quality of the CA matrix. Li et al. [10] combined kernel learning with local information preservation to propose a novel CA matrix learning method. Hao et al. [23] employed feature representations from an encoder to strengthen the structure of the CA matrix.

Third, several studies [14,15] have integrated external knowledge or refinement strategies into CA matrix learning. For example, Zhou et al. [14] enhanced the CA matrix using an active learning strategy to automatically label key samples. Zhou et al. [15] integrated consensus learning with base clustering refinement, thereby enhancing the CA matrix.

Additionally, inspired by the success of hybrid deep learning architectures in capturing complex data patterns and robust feature representations [24], recent works [25,26,27] have been proposed to improve the ensemble clustering performance. For example, Miklautz et al. [25] employed deep learning to generate a set of base clustering results and performed ensemble clustering in the feature space. It extends the utilization of the label space to some extent; however, it still falls short of fully leveraging the rich information embedded in the feature space, such as higher-order relationships between samples. Liang [26] utilized deep learning to produce base clustering results and integrated multiple such outcomes, with the ensemble result subsequently being used to refine the base clustering results. It enables the transfer of final clustering information from the label space back to the feature space, thereby improving clustering performance. Nevertheless, it fails to adequately account for the noise in the connective matrices within the label space and the structural differences among clusters across different matrices. Huang et al. [27] adopted graph convolutional networks (GCNs) to explore high-order structural information embedded in the data and integrated raw data features into the CA matrix. Although this method attempts to combine information from both the feature space and the label space, it still overlooks the noise in the connective matrices and the variations in cluster structures associated with them.

In summary, although the aforementioned ensemble clustering methods have successfully improved the CA matrix and achieved notable clustering performance, they still face the following challenges.

(1) These methods typically directly fuse multiple connective matrices derived from different cluster structures. The resulting CA matrix may suffer from noise introduced by individual connective matrices and structural corruption caused by differences in the cluster structures of these matrices, which may adversely affect the overall performance of ensemble clustering.

(2) These methods primarily exploit information in the label space (i.e., base clustering results). As a result, the CA matrix they construct captures only label space information, neglecting the rich structural and relational information inherent in the feature space (i.e., the original dataset), which may significantly hinder ensemble clustering performance.

To address the aforementioned challenges, we address the following attempts: (1) Due to the symmetry of connective matrices [28], a symmetric error matrix is introduced for each connective matrix to characterize its potential noise distribution. Subsequently, we design a set of mapping models. By assigning a mapping model to each connective matrix, we recover a highly reliable consensus matrix. This process eliminates structural corruption caused by differences in cluster structures among the connective matrices. (2) Multi-order graph structures are designed to explore high-order structural information in the feature space, allowing richer relational patterns among samples to be effectively captured. Based on this, a dynamic combination strategy is further proposed to adaptively integrate information from graph structures of different orders, thereby refining and enhancing the representation capability of the consensus matrix. In addition, a theoretical rank constraint with a block-diagonal structure is imposed on the consensus matrix. Then, we explain our method in detail from the following aspects.

(1) Recent studies [29] have shown that accounting for differences among cluster structures in the connective matrices is essential for learning a reliable consensus matrix. Furthermore, as the connective matrices are generated under different clustering configurations, they can be viewed as observations of the consensus matrix from diverse structural perspectives. However, these observations may be contaminated by noise. Existing studies, such as [28], provide indirect evidence of noise and outliers within the connective matrices. Therefore, an effective strategy is to first denoise the connective matrices and then recover a reliable consensus matrix. Motivated by this, we introduce a symmetric error matrix for each connective matrix to capture its inherent noise. In doing so, the influence of unreliable sample relationships is automatically attenuated during the consensus learning process. This mechanism serves a similar purpose to cluster weighting. Then, we design a set of mapping models to eliminate structural corruption caused by differences among connective matrices, enabling more robust consensus matrix recovery.

(2) The CA matrix captures only the association relationships between pairwise samples within the label space [30]. However, when the label space contains noise or erroneous information, the information in the feature space can serve as a powerful supplement to correct such biases. Therefore, integrating the structural information of samples and the latent cluster structure in the feature space into ensemble clustering is a practical strategy to compensate for deficiencies in the label space and significantly enhance clustering performance. In addition, as noted in [31], the association relationships between pairwise samples are not limited to direct first-order connections but also encompass higher-order connections. Motivated by these observations, we propose multi-order graph structures to incorporate multi-scale connectivity information between pairwise samples from the feature space.

(3) Existing studies [32,33] have demonstrated that the block-diagonal structure plays a critical role in improving clustering performance. Following the idea in [32], we introduce a theoretical rank constraint to enhance the block-diagonal property of the consensus matrix while preserving a clear cluster structure. It is worth noting that the Laplacian rank constraint [18] can also be employed to reinforce the block-diagonal structure of a matrix. However, this Laplacian rank constraint is not a viable option. The reason is that the number of samples $n$ is typically much larger than the number of ground-truth clusters $c$, resulting in a high rank for the Laplacian matrix. Therefore, it is generally unreasonable to seek such a high-rank matrix by minimizing the rank of the Laplacian matrix [33].

In summary, motivated by the aforementioned theories and methods, we propose a novel ensemble clustering method, termed ECM-RCL. Specifically, we first introduce a symmetric error matrix for each connective matrix to identify noise components and then design a set of mapping models to recover the consensus matrix robustly. We further introduce multi-order graph structures to refine the consensus matrix. Finally, a theoretical rank constraint is applied to enhance the block-diagonal structure, thereby more effectively preserving a clear cluster structure.

To describe clearly, here we summarize the differences between the proposed ECM-RCL and some existing methods [3,7,10,12,13,14,15,16,17,18,19,20] from the following six perspectives (e.g., WBC, WC, CSD, CD, BDS, and OD) in

Table 1. The six perspectives of the proposed ECM-RCL and some existing methods.

Methods	WBC	WC	CSD	CD	BDS	OD
Bagherinia et al. [12]	√
Huang et al. [3]		√
Tao et al. [13]				Weak	Weak
Zhou et al. [14]	√				√	√
Jia et al. [7]		√		Weak
Zhou et al. [15]	√					√
Gu et al. [16]	√	√
Xu et al. [17]		√		Weak
Li et al. [10]				Weak
Xu et al. [18]		√		Weak	√
Zheng et al. [20]	√				√
Yang et al. [19]		√			√
ECM-RCL	√		√	√	√	√

. Specifically, WBC indicates whether weighting or selection of base clustering results is considered. WC indicates whether clusters in the base clustering results are weighted. CSD considers differences in cluster structures across connective matrices, but the weighted cluster does not modify its cluster structures. CD indicates whether noise in the connective matrix is considered. BDS denotes whether the block-diagonal property of the CA matrix is considered. OD indicates whether the original dataset information is utilized. In Table 1, “√” suggests that the method finds the corresponding feature, while “Weak” denotes limited or indirect consideration of that feature.

The core contributions of this study can be summarized as follows:

(1)

A symmetric error matrix is introduced for each connective matrix to identify noise components effectively. Additionally, a reliable consensus matrix is recovered by designing a set of mapping models that address structural differences among the connective matrices and enable robust consensus learning.

(2)

Multi-order graph structures are designed to fully exploit the association relationships between samples in the feature space, thereby enhancing the structural quality of the consensus matrix.

(3)

A theoretical rank constraint is incorporated to reinforce the block-diagonal property of the consensus matrix, ensuring a clear cluster structure.

(4)

The experimental results on all adopted datasets demonstrate that ECM-RCL is effective compared to the state-of-the-art methods.

For clarity, the mathematical notations used in this study are summarized in

Table 2. Description of notation.

Notations	Descriptions
$\mathit{X}$	The dataset.
$\mathit{S}$	The CA matrix.
${\mathit{S}}_{pq}$	$\mathrm{The} \left(p,q\right)$-th element in $S$
${\mathit{A}}_i$	The $i$-th connective matrix.
${\mathit{E}}_i$	The $i$-th error matrix.
$\mathit{I}$	The identity matrix.
${\mathit{P}}_i$	The $i$-th mapping matrix.
${\mathit{G}}^j$	The $j$-th order graph structure.
${\mathit{G}}_{pq}^j$	$\mathrm{The} \left(p,q\right)$$\text{-th} \mathrm{element} \mathrm{in} G^j$.
$\mathit{H}$	The consensus matrix.
${\mathit{H}}_{pq}$	$\mathrm{The} \left(p,q\right)$-th element in $\mathit{H}$.
${\pi }^i$	The $i$-th base clustering result.
$m$	The number of base clustering results.
$V$	The maximum order of the multi-order graph structures.
$n$	The number of samples.
$c$	The number of ground-truth clusters.
$c_i$	The number of clusters of the $i$-th base clustering result.
$d$	The number of features in $\mathit{X}$.
${\mathit{x}}_p$	The $p$-th sample.
${‖•‖}_F^2$	The Frobenius norm of $•$.
$tr\left(•\right)$	The trace of $•$.

.

The remainder of this paper is organized as follows: We present preliminaries on ECM-RCL in Section 2. We elaborate further on the ECM-RCL model in Section 3. We present the experimental results of ECM-RCL in Section 4. We conclude with a summary of the findings and future study directions for ECM-RCL in Section 5.

2. Preliminaries

2.1. Co-Association Matrix

The co-association (CA) matrix quantifies association relationships between pairs of samples. Under the theoretical framework in [8], the CA matrix can be systematically derived from a given set of connective matrices (i.e., $\left\{{\mathit{A}}_1,{\mathit{A}}_2,\dots ,{\mathit{A}}_m\right\}$).

$$\mathit{S}={\displaystyle \frac{1}{m}}{\displaystyle \sum _{i=1}^m{\mathit{A}}_i}$$

(1)

$${\mathit{A}}_i=\beta \left({\mathit{cls}}^i\left({\mathit{x}}_p\right),{\mathit{cls}}^i\left({\mathit{x}}_q\right)\right)$$

(2)

where ${\mathit{cls}}^i\left({\mathit{x}}_p\right)$ denotes the cluster membership of ${\mathit{x}}_{\mathrm{p}}$ in the $i$-th base clustering result ${\pi }^{\mathrm{i}}$, and $\beta \left(a,b\right)$ is the Kronecker delta function, which can be defined as follows:

$$\beta (a,b)=\left\{\begin{array}{l}1,a=b\hfill \\ 0,a\ne b\hfill \end{array}\right.$$

(3)

2.2. Multi-Order Graph Structures

First-order proximity is a fundamental concept in graph analysis, capturing the direct relationships between samples as indicated by adjacency information.

$${\mathit{G}}_{pq}^1=\left\{\begin{array}{l}\kappa \left({\mathit{x}}_p,{\mathit{x}}_q\right),{\mathit{x}}_q\in N_K\left({\mathit{x}}_p\right)\hfill \\ 0,\mathrm{otherwise}\hfill \end{array}\right.$$

(4)

The parameter $K$ denotes the number of nearest neighbors. Although the first-order graph structure captures direct similarities, it cannot reveal global structural patterns. To overcome this, higher-order structures leverage shared neighbors to uncover latent associations among samples [34]. The specific construction method is as follows:

$${\mathit{G}}^j=\left\{\begin{array}{l}{\mathit{G}}^1,j=1\hfill \\ {\mathit{G}}^{j-1}{\mathit{G}}^1,j>1\hfill \end{array}\right.$$

(5)

Multi-order graph structures integrate both first-order and higher-order proximities, systematically capturing multi-level relationships among samples and thereby fully exploiting the structural information present in the feature space.

3. Ensemble Clustering Method via Robust Consensus Learning

In this section, we provide a detailed description of ECM-RCL, as illustrated in Figure 1. First, for each connective matrix, a symmetric error matrix is constructed to characterize potential noise. Next, a set of mapping models is designed to reliably recover the consensus matrix. Second, multi-order graph structures are built in the feature space to capture the rich structural information embedded in the samples, further refining the consensus matrix. Third, a block-diagonal enhancement constraint is incorporated to preserve the clear structural properties of the consensus matrix. Finally, spectral clustering is applied to the refined consensus matrix to obtain the final clustering result.

3.1. The Objective Function of ECM-RCL

In this subsection, we formally introduce the objective function of ECM-RCL. As discussed in the preceding sections, ensemble clustering methods based on the CA matrix still face two critical but often overlooked challenges. First, during the consensus matrix learning process, these methods commonly ignore noise contamination within individual connective matrices and structural differences among connective matrices. It is worth noting that although cluster weighting can enhance the influence of reliable clusters, it cannot effectively modify the inherent structural differences among connective matrices. Second, most existing methods focus primarily on the label space, overlooking the rich structural and relational information embedded in the feature space.

In the label space, given a set of connective matrices (i.e., $\left\{{\mathit{A}}_1,{\mathit{A}}_2,\dots ,{\mathit{A}}_m\right\}$), the first step is to recover a reliable consensus matrix from them. To address the noise within each connective matrix, we leverage its symmetry [28]. Adding to this, we introduce a symmetric error matrix ${\mathit{E}}_i$ for each matrix to account for potential observation noise. The denoised connective matrices are thus obtained via error compensation (i.e., $\left\{{\mathit{A}}_1-{\mathit{E}}_1,{\mathit{A}}_2-{\mathit{E}}_2,\dots ,{\mathit{A}}_m-{\mathit{E}}_m\right\}$). Subsequently, we propose a set of mapping models (i.e., $\left\{{\mathit{P}}_1,{\mathit{P}}_2,\dots ,{\mathit{P}}_m\right\}$) to address differences in cluster structures across connective matrices. Each denoised matrix is assigned to a mapping model and projected into a unified space for consensus matrix recovery.

$$\begin{array}{l}\underset{{\mathit{P}}_i,\mathit{H},{\alpha }_i}{\min }{\displaystyle \sum _{i=1}^m{\alpha }_i^2{‖{\mathit{A}}_i-{\mathit{E}}_i-{\mathit{P}}_i\mathit{H}‖}_F^2}+{\lambda }_1{\displaystyle \sum _{i=1}^m{‖{\mathit{E}}_i‖}_F^2}+{\lambda }_3{\displaystyle \sum _{i=1}^m{‖{\mathit{P}}_i‖}_F^2}\\ \mathrm{s}.\mathrm{t}.{\mathit{E}}_i={\mathit{E}}_i^T,\mathit{H}={\mathit{H}}^T,{\displaystyle \sum _{i=1}^m{\alpha }_i=1},0\le {\alpha }_i\le 1\end{array}$$

(6)

where ${\lambda }_1$ and ${\lambda }_3$ are hyperparameters, ${\alpha }_i$ denotes the weight assigned to the $i$-th connective matrix.

The consensus matrix obtained from the label space alone may not sufficiently represent feature space structures, limiting clustering performance. To enhance its expressiveness, we integrate it with structural information from the feature space. Our goal is to learn a unified consensus matrix that jointly captures cluster structures from both spaces. Accordingly, the objective function of ECM-RCL is defined as

$$\begin{array}{l}\underset{{\mathit{P}}_i,\mathit{H},{\mathit{E}}_i,{\alpha }_i,{\beta }_j}{\min }{\displaystyle \sum _{i=1}^m{\alpha }_i^2{‖{\mathit{A}}_i-{\mathit{E}}_i-{\mathit{P}}_i\mathit{H}‖}_F^2}+{\lambda }_1{\displaystyle \sum _{i=1}^m{‖{\mathit{E}}_i‖}_F^2}+{\lambda }_2{\displaystyle \sum _{j=1}^V{\beta }_j^2{‖\mathit{H}-{\mathit{G}}^j‖}_F^2}+{\lambda }_3{\displaystyle \sum _{i=1}^m{‖{\mathit{P}}_i‖}_F^2}\\ \mathrm{s}.\mathrm{t}.0\le {\mathit{H}}_{pq}\le 1,\mathit{H}={\mathit{H}}^T,tr(\mathit{H})=c,{\mathit{E}}_i={{\mathit{E}}_i}^T,{\displaystyle \sum _{i=1}^m{\alpha }_i=1},0\le {\alpha }_i\le 1,{\displaystyle \sum _{\mathrm{j}=1}^{\mathrm{V}}{\beta }_j=1},0\le {\beta }_j\le 1\end{array}$$

(7)

where ${\lambda }_2$ is a hyperparameter. The theoretical rank constraint $tr(\mathit{H})=c$ approximates the graph structure by preserving the block-diagonal property of the matrix. The constraint can help preserve a clear cluster structure. ${\beta }_j$ denotes the weight assigned to the $j$-th order graph structure.

After obtaining the consensus matrix, spectral clustering is applied to generate the final ensemble clustering result. The spectral clustering method has two main advantages. First, the consensus matrix $\mathit{H}$ can directly serve as input for spectral clustering. Second, $\mathit{H}$ exhibits a block-diagonal structure that aligns well with the structural requirements of an ideal similarity matrix in spectral clustering [33].

3.2. Optimization and Analysis

3.2.1. Optimization

In this subsection, we optimize the objective function in Equation (7).

Step 1: ${\mathit{P}}_i$ subproblem

When optimizing $P_i$, we fix the other variables, and the objective function Equation (7) reduces to

$$\underset{{\mathit{P}}_i}{\min }{\alpha }_i^2{‖{\mathit{A}}_i-{\mathit{E}}_i-{\mathit{P}}_{\mathrm{i}}\mathit{H}‖}_F^2+{\lambda }_3{‖{\mathit{P}}_{\mathrm{i}}‖}_F^2$$

(8)

Given that Equation (8) is an unconstrained convex optimization problem, its global minimum can be efficiently found by setting the gradient of the objective function with respect to ${\mathit{P}}_i$ to zero.

$${\mathit{P}}_i=\left({\alpha }_i^2\left({\mathit{A}}_i-{\mathit{E}}_i\right){\mathit{H}}^T\right){\left({\alpha }_i^2\mathit{H}{\mathit{H}}^T+{\lambda }_3\mathit{I}\right)}^{-1}$$

(9)

Step 2: $\mathit{H}$ subproblem

When optimizing $\mathit{H}$, we fix the other variables, and the objective function Equation (7) reduces to

$$\begin{array}{l}\underset{\mathit{H}}{\min }{\displaystyle \sum _{i=1}^m{\alpha }_i^2{‖{\mathit{A}}_i-{\mathit{E}}_i-{\mathit{P}}_i\mathit{H}‖}_F^2}+{\lambda }_2{\displaystyle \sum _{j=1}^V{\beta }_j^2{‖\mathit{H}-{\mathit{G}}^j‖}_F^2}\\ \mathrm{s}.\mathrm{t}.0\le {\mathit{H}}_{pq}\le 1,\mathit{H}={\mathit{H}}^T,tr(\mathit{H})=c\end{array}$$

(10)

Since direct optimization is challenging, we adopt a two-step approximation strategy based on [35] to optimize Equation (10). Specifically, we first represent $\mathit{H}$ as $\tilde{\mathit{H}}$, disregarding the constraints on $\mathit{H}$, and then incorporate these constraints to adjust $\tilde{\mathit{H}}$ into an approximation of $\mathit{H}$. We transform the problem in Equation (10) into the problems in Equations (11) and (12).

$$\underset{\tilde{\mathit{H}}}{\min }{\displaystyle \sum _{i=1}^m{\alpha }_i^2{‖{\mathit{A}}_i-{\mathit{E}}_i-{\mathit{P}}_i\tilde{\mathit{H}}‖}_F^2}+{\lambda }_2{\displaystyle \sum _{j=1}^V{\beta }_j^2{‖\tilde{\mathit{H}}-{\mathit{G}}^j‖}_F^2}$$

(11)

$$\begin{array}{l}\underset{\mathit{H}}{\min }{‖\mathit{H}-\tilde{\mathit{H}}‖}_F^2\\ \mathrm{s}.\mathrm{t}.0\le {\mathit{H}}_{pq}\le 1,\mathit{H}={\mathit{H}}^T,tr(\mathit{H})=c\end{array}$$

(12)

Given that Equation (11) is an unconstrained convex optimization problem, its global minimum can be efficiently found by setting the gradient of the objective function with respect to $\tilde{\mathit{H}}$ to zero.

$$\tilde{\mathit{H}}={\left({\displaystyle \sum _{i=1}^m{\alpha }_i^2{\mathit{P}}_i^T{\mathit{P}}_i+{\lambda }_2{\displaystyle \sum _{j=1}^V{\beta }_j^2\mathit{I}}}\right)}^{-1}\left({\displaystyle \sum _{i=1}^m{\alpha }_i^2{\mathit{P}}_i^T\left({\mathit{A}}_i-{\mathit{E}}_i\right)+{\lambda }_2{\displaystyle \sum _{j=1}^V{\beta }_j^2{\mathit{G}}^j}}\right)$$

(13)

When we obtain $\tilde{\mathit{H}}$, we can directly derive that $\mathit{H}$ is solved by the manner [32].

Step 3: ${\mathit{E}}_i$ subproblem

When optimizing ${\mathit{E}}_i$, we fix the other variables, and the objective function Equation (7) reduces to

$$\begin{array}{l}\underset{{\mathit{E}}_i}{\min }{\alpha }_i^2{‖{\mathit{A}}_i-{\mathit{E}}_i-{\mathit{P}}_i\mathit{H}‖}_F^2+{\lambda }_1{‖{\mathit{E}}_i‖}_F^2\\ \mathrm{s}.\mathrm{t}.{\mathit{E}}_i={{\mathit{E}}_i}^T\end{array}$$

(14)

According to [7], the objective function in Equation (14) is recast as Equation (15).

$$\begin{array}{l}\underset{{\mathit{E}}_i}{\min }{‖{\mathit{E}}_i-{\displaystyle \frac{{\alpha }_i^2\left({\mathit{A}}_i-{\mathit{P}}_i\mathit{H}\right)}{{\alpha }_i^2+{\lambda }_1}}‖}_F^2\\ \mathrm{s}.\mathrm{t}.{\mathit{E}}_i={{\mathit{E}}_i}^T\end{array}$$

(15)

Although Equation (15) is a constrained optimization problem, it can be effectively solved in the manner of [7].

$${\mathit{E}}_i={\displaystyle \frac{{\alpha }_i^2\left(\left({\mathit{A}}_i-{\mathit{P}}_i\mathit{H}\right)+{\left({\mathit{A}}_i-{\mathit{P}}_i\mathit{H}\right)}^T\right)}{2\left({\alpha }_i^2+{\lambda }_1\right)}}$$

(16)

Step 4: ${\alpha }_{\mathrm{i}}$ subproblem

When optimizing ${\alpha }_{\mathrm{i}}$, we fix the other variables, and the objective function Equation (7) reduces to

$$\begin{array}{l}\underset{{\alpha }_i}{\min }{\displaystyle \sum _{i=1}^m{\alpha }_i^2{‖{\mathit{A}}_i-{\mathit{E}}_i-{\mathit{P}}_i\mathit{H}‖}_F^2}\\ \mathrm{s}.\mathrm{t}.{\displaystyle \sum _{i=1}^m{\alpha }_i=1},0\le {\alpha }_i\le 1\end{array}$$

(17)

We derive the solution for ${\alpha }_i$ based on the Cauchy–Schwarz inequality.

$${\alpha }_i={\displaystyle \frac{{‖{\mathit{A}}_i-{\mathit{E}}_i-{\mathit{P}}_i\mathit{H}‖}_F^{-2}}{{\displaystyle \sum _{i=1}^m{‖{\mathit{A}}_i-{\mathit{E}}_i-{\mathit{P}}_i\mathit{H}‖}_F^{-2}}}}$$

(18)

For details on the application of the Cauchy–Schwarz inequality to weight allocation, refer to Appendix A.

Step 5: ${\beta }_j$ subproblem

When optimizing ${\beta }_j$, we fix the other variables, and the objective function Equation (7) reduces to

$$\begin{array}{l}\underset{{\beta }_j}{\min }{\displaystyle \sum _{j=1}^V{\beta }_j^2{‖\mathit{H}-{\mathit{G}}^j‖}_F^2}\\ \mathrm{s}.\mathrm{t}.{\displaystyle \sum _{j=1}^V{\beta }_j}=1,0\le {\beta }_j\le 1\end{array}$$

(19)

We derive the solution for ${\beta }_j$ based on the Cauchy–Schwarz inequality.

$${\beta }_j={\displaystyle \frac{{‖\mathit{H}-{\mathit{G}}^j‖}_F^{-2}}{{\displaystyle \sum _{j=1}^V{‖\mathit{H}-{\mathit{G}}^j‖}_F^{-2}}}}$$

(20)

For details on the application of the Cauchy–Schwarz inequality to weight allocation, refer to Appendix A.

The optimization process and the overall workflow of ECM-RCL are summarized in Algorithms 1 and 2, respectively.

Algorithm 1: Optimization of Objective Function

Input:$\mathrm{Connective} \mathrm{matrices} \left\{{\mathit{A}}_1,{\mathit{A}}_2,\dots ,{\mathit{A}}_m\right\}$, multi-order graph structures $\left\{{\mathit{G}}^1,{\mathit{G}}^2,\dots ,{\mathit{G}}^V\right\}$$, \mathrm{hyperparameters} {\lambda }_1$$, {\lambda }_2$$\mathrm{and} {\lambda }_3$.

Initialize: $iter=0$$, {\alpha }_i={\displaystyle \frac{1}{m}}$$, {\beta }_j={\displaystyle \frac{1}{V}}$$, {\mathit{E}}_i=\mathit{0}$$, \mathit{H}=\mathit{0}$$, {\mathit{P}}_i=\mathit{I}$.     $\mathrm{Obtain} {\mathit{P}}_i$ by Equation (9).     $\mathrm{Obtain} \mathit{H}$ by Equation (10). while not converging do     $iter=iter+1$     $\mathrm{Update} {\mathit{E}}_i$ by Equation (16).     $\mathrm{Update} {\mathit{P}}_i$ by Equation (9).     $\mathrm{Update} \mathit{H}$ by Equation (10).     $\mathrm{Update} {\alpha }_i$ by Equation (18).     $\mathrm{Update} {\beta }_j$ by Equation (20).     Check the convergence conditions:        $\left|\mathrm{obj}\left(iter\right)-\mathrm{obj}\left(iter-1\right)\right|<1 \times {10}^{-4}\ast \mathrm{obj}\left(iter\right)$ end while

Output: Consensus Matrix $\mathit{H}$

Algorithm 2: ECM-RCL

Input:$\mathrm{Dataset} \mathit{X}\in {\mathit{R}}^{n\times d}$, the number of base clustering results $m$, the number of ground-truth clusters $c$, the number of clusters of base clustering results $\left\{c_1,c_2,\dots ,c_m\right\}$.

Step 1: Obtain $m$ connective matrices $\left\{{\mathit{A}}_1,{\mathit{A}}_2,\dots ,{\mathit{A}}_m\right\}$. Step 2: Obtain multi-order graph structures $\left\{{\mathit{G}}^1,{\mathit{G}}^2,\dots ,{\mathit{G}}^V\right\}$. Step 3: Call Algorithm 1 to obtain the consensus matrix $\mathit{H}$. Step 4: Obtain clustering results by spectral clustering.

Output: Final ensemble clustering result.

3.2.2. Complexity Analysis

This subsection analyzes the time complexity of ECM-RCL. According to Algorithm 2, it comprises four steps:

Step 1 aims to generate connective matrices. In this study, the K-means method is employed to produce the base clustering results, from which the corresponding connectivity matrices are constructed. Assuming that all K-means method processes have the same number of iterations, the time complexity of this step is dominated by the complexity of K-means (i.e., $\max \left(\mathrm{O}\left(n{c}_{i}dt\right)\right)$), where $t$ denotes the number of iterations.

Step 2 involves constructing multi-order graph structures. The time complexity of building the first-order graph structure is $\mathrm{O}\left(n^3\right)$, while the overall time complexity of generating the multi-order graph structures is $\mathrm{O}\left(V{n}^3\right)$.

Step 3 corresponds to the optimization process in Algorithm 1. The time complexities for updating ${\left\{{\mathit{P}}_i\right\}}_{i=1}^m$, $\mathit{H}$ and ${\left\{{\mathit{E}}_i\right\}}_{i=1}^m$ require $\mathrm{O}\left(m{n}^3\right)$. It needs $\mathrm{O}\left(m{n}^2\right)$ to update ${\left\{{\alpha }_i\right\}}_{i=1}^m$. It needs $\mathrm{O}\left(V{n}^2\right)$ to update ${{\left\{\beta \right\}}_{j=1}}^V$. It reveals that the time complexity of a key step in this algorithm is $\mathrm{O}\left(n^3\right)$, a cost arising primarily from the iterative updates of parameters ${\left\{{\mathit{P}}_i\right\}}_{i=1}^m$ and $\mathit{H}$. As the sample size $n$ increases, the computational burden grows significantly. Compared with the improvement in robustness and accuracy of ECM-RCL on large-scale datasets, the computational burden is worthwhile. Additionally, some methods (e.g., Nyström method [36] or anchor-based method [37]) can be adopted to further reduce the computational burden.

Step 4 corresponds to obtaining the final clustering result using spectral clustering. The time complexity of spectral clustering is $\mathrm{O}\left(n^3+c{n}^2\right)$.

In summary, the total time cost of ECM-RCL is $\max \left(\mathrm{O}\left(n{c}_{i}dt \right)\right)+\mathrm{O}\left((V+m)n^3\right)$.

We observe that as the sample size increases, the time complexity of the method shows a significant upward trend. Meanwhile, the clustering performance of the proposed model also improves. Although the computational cost increases, the rise in time complexity remains acceptable given the performance gains achieved. In subsequent research, optimization strategies such as the introduction of the Nyström method or the anchor-based method can be explored to effectively reduce time complexity.

4. Experimental Section

In this section, extensive experiments are conducted to evaluate the effectiveness of the proposed ECM-RCL. To ensure fairness, all experiments are performed on a machine equipped with a 14th-generation Intel Core i7-14700KF processor and 64 GB of RAM.

4.1. Experimental Organization

4.1.1. Datasets

To systematically evaluate the clustering performance of the proposed ECM-RCL, 19 real-world benchmark datasets were selected. The details of each dataset are summarized in

Table 3. The details of the datasets.

Datasets	Number of Samples	Number of Features	Number of Clusters
ALLAML	72	7129	2
CLL_SUB	111	11,340	3
GLIMA	50	4434	4
Heart	270	13	2
Iris	150	4	3
LM	360	90	15
Lymphoma	96	4026	9
MF	2000	649	10
Orlraws10p	100	10,304	10
USPS	1854	256	10
Vertebral	310	6	3
WarpPIE10p	210	2420	10
WDBC	569	30	2
Wine	178	13	3
Zoo	101	16	7
ISOLET	7797	617	26
LS	6435	36	6
ODR	5620	64	10
PD	10,992	16	10

.

4.1.2. Comparative Methods

In this subsection, we introduce the seven comparative methods: K-means clustering method (K-means) [11], Locally Weighted Ensemble Clustering (LWEA) [3], Ensemble Clustering Based on Dense Representation (DREC) [38], Enhanced Ensemble Clustering via Fast Propagation of Cluster-Wise Similarities (ECPCS-HC) [4], Ensemble Clustering via Co-Association Matrix Self-Enhancement (EC-CMS) [7], Ensemble Clustering with Attentional Representation (ECAR) [23], and Clustering Ensemble via Diffusion on Adaptive Multiplex (CEAM) [39].

4.1.3. Experimental Setup and Evaluation Indices

In this study, following the common practice in the literature [40], we adopted K-means as the base clustering method and set the number of base clustering results to 10 (i.e., $m=10$). For each base clustering, the number of clusters was randomly selected from the range $\left[c,\max \left(100,\sqrt{n}\right)\right]$. For the comparative methods, all parameter settings followed the configurations described in [3,4,7,23,38,39]. All experimental results were averaged over 10 independent runs.

In the ECM-RCL, the number of neighbors $K$ for constructing the k-nearest neighbor graph is set to $\min \left(10,{\displaystyle \frac{n}{2c}}\right)$ when the number of samples is less than 1000 ($n<1000$), and $\min \left(20,{\displaystyle \frac{n}{2c}}\right)$ otherwise. The maximum order of the multi-order graph structures is set to $2$ ($V=2$). The parameters of ECM-RCL are determined via grid search, where the candidate values for ${\lambda }_1$, ${\lambda }_2$, and ${\lambda }_3$ are $\left\{0.001,0.01,0.1,1,10,100,1000\right\}$.

In addition, three commonly used evaluation metrics are employed to assess the clustering performance of the proposed method: Accuracy (ACC) [41], Normalized Mutual Information (NMI) [42], and Adjusted Rand Index (ARI) [43]. The values of ACC and NMI range from $\left[0,1\right]$, while ARI ranges from $\left[-1,1\right]$. Higher values indicate better clustering performance.

4.2. Discussions on Datasets

4.2.1. Clustering Performance

In this section, we compare the clustering performance of the proposed ECM-RCL with seven clustering methods on all the adopted datasets. The average ACC, NMI, and ARI scores for all adopted methods across all datasets are presented in

Table 4. Average ACC of all the adopted methods on all adopted datasets.

Datasets	K-Means	LWEA	DREC	ECPCS-HC	EC-CMS	ECAR	CEAM	ECM-RCL
ALLAML	0.6903 ± 0.0371	0.6833 ± 0.0667	0.6903 ± 0.0687	0.6569 ± 0.0365	0.6722 ± 0.0736	0.6806 ± 0.0000	0.5972 ± 0.0729	0.7153 ± 0.0150
CLL_SUB	0.5252 ± 0.0085	0.5279 ± 0.0047	0.5279 ± 0.0047	0.5279 ± 0.0047	0.5270 ± 0.0047	0.4775 ± 0.0000	0.4486 ± 0.0337	0.5459 ± 0.0047
GLIOMA	0.5600 ± 0.0794	0.5900 ± 0.0216	0.5860 ± 0.0232	0.5740 ± 0.0212	0.5900 ± 0.0368	0.3000 ± 0.0000	0.5680 ± 0.0473	0.6440 ± 0.0430
Heart	0.5904 ± 0.0019	0.5856 ± 0.0271	0.6037 ± 0.0287	0.5933 ± 0.0298	0.6100 ± 0.0243	0.6185 ± 0.0000	0.5837 ± 0.0379	0.6244 ± 0.0053
Iris	0.8560 ± 0.1041	0.8747 ± 0.0301	0.9020 ± 0.0336	0.8993 ± 0.0320	0.8987 ± 0.0320	0.6667 ± 0.0000	0.7300 ± 0.0668	0.9140 ± 0.0685
LM	0.4406 ± 0.0267	0.4503 ± 0.0153	0.4558 ± 0.0117	0.4292 ± 0.0184	0.4550 ± 0.0194	0.4611 ± 0.0000	0.4583 ± 0.0154	0.5200 ± 0.0140
Lymphoma	0.5427 ± 0.0546	0.5740 ± 0.0724	0.5646 ± 0.0576	0.6948 ± 0.0908	0.5198 ± 0.0700	0.5000 ± 0.0000	0.5677 ± 0.0256	0.6552 ± 0.0421
MF	0.5036 ± 0.0453	0.5493 ± 0.0298	0.5666 ± 0.0338	0.5447 ± 0.0359	0.6057 ± 0.0302	0.5200 ± 0.0000	0.5751 ± 0.0647	0.8004 ± 0.0096
Orlraws10P	0.6730 ± 0.0536	0.7960 ± 0.0481	0.7830 ± 0.0447	0.7220 ± 0.0377	0.7980 ± 0.0424	0.6500 ± 0.0000	0.8010 ± 0.0567	0.8470 ± 0.0408
USPS	0.6182 ± 0.0278	0.6836 ± 0.0254	0.6901 ± 0.0372	0.6859 ± 0.0560	0.7273 ± 0.0453	0.6208 ± 0.0000	0.6972 ± 0.0471	0.7573 ± 0.0000
Vertebral	0.5984 ± 0.0546	0.6177 ± 0.1135	0.5352 ± 0.0284	0.6110 ± 0.1337	0.5358 ± 0.0195	0.6516 ± 0.0000	0.5652 ± 0.0680	0.6703 ± 0.0866
WarpPIE10p	0.2605 ± 0.0182	0.2357 ± 0.0093	0.2552 ± 0.0202	0.2124 ± 0.0068	0.2386 ± 0.0182	0.1952 ± 0.0000	0.2676 ± 0.0214	0.4762 ± 0.0135
WDBC	0.8541 ± 0.0000	0.8035 ± 0.0876	0.8283 ± 0.0296	0.8738 ± 0.0411	0.8190 ± 0.0763	0.9244 ± 0.0000	0.8649 ± 0.0335	0.8903 ± 0.0191
Wine	0.6292 ± 0.0644	0.6713 ± 0.0651	0.6809 ± 0.0388	0.6494 ± 0.0746	0.6691 ± 0.0701	0.6404 ± 0.0000	0.6758 ± 0.0464	0.7247 ± 0.0000
Zoo	0.7347 ± 0.0800	0.7426 ± 0.0585	0.7455 ± 0.0469	0.7436 ± 0.0211	0.7188 ± 0.0725	0.7822 ± 0.0000	0.6267 ± 0.0841	0.7248 ± 0.0283
ISOLET	0.5255 ± 0.0284	0.5620 ± 0.0100	0.5624 ± 0.0163	0.5226 ± 0.0119	0.5508 ± 0.0015	0.5256 ± 0.0000	0.5319 ± 0.0107	0.5645 ± 0.0023
LS	0.6342 ± 0.0678	0.6276 ± 0.1915	0.6494 ± 0.0026	0.6709 ± 0.1789	0.6253 ± 0.0065	0.7052 ± 0.0000	0.5460 ± 0.1282	0.7239 ± 0.0036
ODR	0.7592 ± 0.0649	0.8391 ± 0.0316	0.9017 ± 0.0457	0.8669 ± 0.0025	0.9212 ± 0.0045	0.7194 ± 0.0000	0.8484 ± 0.0033	0.9790 ± 0.0000
PD	0.7039 ± 0.0494	0.7836 ± 0.0031	0.8088 ± 0.0519	0.7426 ± 0.0270	0.7832 ± 0.0060	0.6543 ± 0.0000	0.7212 ± 0.0384	0.8890 ± 0.0021

,

Table 5. Average NMI of all the adopted methods on all adopted datasets.

Datasets	K-Means	LWEA	DREC	ECPCS-HC	EC-CMS	ECAR	CEAM	ECM-RCL
ALLAML	0.0897 ± 0.0393	0.1119 ± 0.0333	0.1209 ± 0.0310	0.0417 ± 0.0452	0.0857 ± 0.0656	0.1003 ± 0.0000	0.0505 ± 0.0379	0.1448 ± 0.0173
CLL_SUB	0.1804 ± 0.0004	0.1805 ± 0.0003	0.1805 ± 0.0003	0.1805 ± 0.0003	0.1804 ± 0.0003	0.0967 ± 0.0000	0.0686 ± 0.0289	0.2627 ± 0.0005
GLIOMA	0.4257 ± 0.1202	0.4921 ± 0.0226	0.4943 ± 0.0267	0.4780 ± 0.0164	0.4938 ± 0.0293	0.0000 ± 0.0000	0.4044 ± 0.0449	0.5253 ± 0.0198
Heart	0.0187 ± 0.0007	0.0176 ± 0.0134	0.0280 ± 0.0136	0.0210 ± 0.0151	0.0295 ± 0.0133	0.0410 ± 0.0000	0.0214 ± 0.0156	0.0409 ± 0.0042
Iris	0.7204 ± 0.0669	0.7505 ± 0.0437	0.7915 ± 0.0528	0.7870 ± 0.0482	0.7862 ± 0.0483	0.7337 ± 0.0000	0.5398 ± 0.0878	0.8131 ± 0.0853
LM	0.5623 ± 0.0243	0.5906 ± 0.0129	0.5930 ± 0.0148	0.5612 ± 0.0176	0.5961 ± 0.0172	0.5801 ± 0.0000	0.5862 ± 0.0171	0.6460 ± 0.0082
Lymphoma	0.5607 ± 0.0424	0.6141 ± 0.0423	0.6161 ± 0.0483	0.6964 ± 0.0509	0.5851 ± 0.0358	0.5575 ± 0.0000	0.5926 ± 0.0174	0.6711 ± 0.0179
MF	0.5594 ± 0.0182	0.6045 ± 0.0252	0.6152 ± 0.0321	0.6151 ± 0.0245	0.6462 ± 0.0233	0.6184 ± 0.0000	0.5896 ± 0.0344	0.7644 ± 0.0083
Orlraws10P	0.7607 ± 0.0323	0.8484 ± 0.0337	0.8437 ± 0.0310	0.8112 ± 0.0304	0.8392 ± 0.0286	0.7846 ± 0.0000	0.8374 ± 0.0413	0.9207 ± 0.0158
USPS	0.6138 ± 0.0150	0.6757 ± 0.0210	0.6797 ± 0.0244	0.6655 ± 0.0180	0.6981 ± 0.0154	0.6405 ± 0.0000	0.6734 ± 0.0283	0.7998 ± 0.0000
Vertebral	0.3898 ± 0.0299	0.3658 ± 0.1729	0.4404 ± 0.1083	0.2954 ± 0.1938	0.2808 ± 0.1750	0.4209 ± 0.0000	0.3926 ± 0.0902	0.4970 ± 0.0576
WarpPIE10p	0.2448 ± 0.0322	0.2115 ± 0.0287	0.2412 ± 0.0291	0.1726 ± 0.0136	0.2210 ± 0.0321	0.1400 ± 0.0000	0.2676 ± 0.0242	0.5562 ± 0.0127
WDBC	0.4223 ± 0.0000	0.3262 ± 0.1703	0.3666 ± 0.0639	0.4696 ± 0.0911	0.3549 ± 0.1524	0.6064 ± 0.0000	0.4313 ± 0.0872	0.5045 ± 0.0441
Wine	0.4100 ± 0.0165	0.4118 ± 0.0264	0.4032 ± 0.0424	0.4083 ± 0.0317	0.4046 ± 0.0365	0.2973 ± 0.0000	0.3546 ± 0.0960	0.4225 ± 0.0116
Zoo	0.7179 ± 0.0521	0.7287 ± 0.0434	0.7254 ± 0.0348	0.7068 ± 0.0498	0.7166 ± 0.0497	0.7921 ± 0.0000	0.6485 ± 0.0578	0.7219 ± 0.0093
ISOLET	0.7114 ± 0.0123	0.7334 ± 0.0094	0.7468 ± 0.0139	0.7101 ± 0.0110	0.7362 ± 0.0067	0.6674 ± 0.0000	0.7232 ± 0.0002	0.7633 ± 0.0015
LS	0.5485 ± 0.0682	0.4937 ± 0.1476	0.6157 ± 0.0120	0.5290 ± 0.1467	0.5215 ± 0.0132	0.6366 ± 0.0000	0.4477 ± 0.1125	0.6732 ± 0.0004
ODR	0.7274 ± 0.0310	0.8206 ± 0.012	0.8565 ± 0.0298	0.8323 ± 0.0016	0.8628 ± 0.0048	0.7045 ± 0.0000	0.8154 ± 0.0052	0.9505 ± 0.0000
PD	0.6712 ± 0.0205	0.7581 ± 0.0443	0.7965 ± 0.0046	0.7327 ± 0.0137	0.7742 ± 0.0161	0.6945 ± 0.0000	0.7265 ± 0.0217	0.8522 ± 0.0051

and

Table 6. Average ARI of all the adopted methods on all adopted datasets.

Datasets	K-Means	LWEA	DREC	ECPCS-HC	EC-CMS	ECAR	CEAM	ECM-RCL
ALLAML	0.1333 ± 0.0596	0.1329 ± 0.0836	0.1451 ± 0.0868	0.0372 ± 0.0797	0.1044 ± 0.1106	0.1189 ± 0.0000	0.0413 ± 0.0650	0.1754 ± 0.0256
CLL_SUB	0.0933 ± 0.0209	0.0867 ± 0.0008	0.0867 ± 0.0008	0.0867 ± 0.0008	0.0866 ± 0.0008	0.0436 ± 0.0000	0.0270 ± 0.0227	0.1230 ± 0.0011
GLIOMA	0.3004 ± 0.1308	0.3588 ± 0.0348	0.3566 ± 0.0390	0.3876 ± 0.0382	0.3760 ± 0.0434	0.0000 ± 0.0000	0.2467 ± 0.0333	0.3854 ± 0.0311
Heart	0.0287 ± 0.0013	0.0266 ± 0.0207	0.0411 ± 0.0227	0.0318 ± 0.0245	0.0459 ± 0.0194	0.0527 ± 0.0000	0.0292 ± 0.0257	0.0584 ± 0.0052
Iris	0.6911 ± 0.0946	0.7016 ± 0.0517	0.7548 ± 0.0736	0.7489 ± 0.0688	0.7476 ± 0.0690	0.5681 ± 0.0000	0.4603 ± 0.1019	0.7975 ± 0.1174
LM	0.2930 ± 0.0273	0.3283 ± 0.0218	0.3262 ± 0.0183	0.3252 ± 0.0245	0.3432 ± 0.0245	0.3076 ± 0.0000	0.3123 ± 0.0223	0.3895 ± 0.0122
Lymphoma	0.3214 ± 0.0893	0.3547 ± 0.0733	0.3426 ± 0.0580	0.5594 ± 0.1150	0.3012 ± 0.0704	0.3270 ± 0.0000	0.3064 ± 0.0263	0.4119 ± 0.0433
MF	0.4242 ± 0.0232	0.4652 ± 0.0304	0.4762 ± 0.0409	0.4823 ± 0.0277	0.5128 ± 0.0307	0.4293 ± 0.0000	0.4436 ± 0.0361	0.6819 ± 0.0072
Orlraws10P	0.5703 ± 0.0541	0.7080 ± 0.0644	0.6986 ± 0.0607	0.6244 ± 0.0589	0.6959 ± 0.0419	0.4328 ± 0.0000	0.7094 ± 0.0732	0.8330 ± 0.0218
USPS	0.5182 ± 0.0228	0.5790 ± 0.0255	0.5760 ± 0.0411	0.5963 ± 0.0480	0.6354 ± 0.0488	0.5347 ± 0.0000	0.5790 ± 0.0451	0.7061 ± 0.0000
Vertebral	0.3145 ± 0.0089	0.3105 ± 0.2637	0.2952 ± 0.1225	0.2585 ± 0.3133	0.1374 ± 0.1809	0.3240 ± 0.0000	0.3198 ± 0.1127	0.4652 ± 0.1460
WarpPIE10p	0.0592 ± 0.0185	0.0381 ± 0.0161	0.0551 ± 0.0191	0.0208 ± 0.0067	0.0407 ± 0.0182	0.0133 ± 0.0000	0.0788 ± 0.0157	0.3181 ± 0.0129
WDBC	0.4914 ± 0.0000	0.3746 ± 0.2025	0.4199 ± 0.0812	0.5568 ± 0.1219	0.4100 ± 0.1872	0.7182 ± 0.0000	0.5294 ± 0.1024	0.6050 ± 0.0615
Wine	0.3575 ± 0.0126	0.3705 ± 0.0191	0.3569 ± 0.0297	0.3563 ± 0.0286	0.3608 ± 0.0414	0.2745 ± 0.0000	0.3320 ± 0.0871	0.4007 ± 0.0000
Zoo	0.6281 ± 0.1125	0.6390 ± 0.0789	0.6405 ± 0.0695	0.6226 ± 0.0501	0.6117 ± 0.0977	0.7178 ± 0.0000	0.5097 ± 0.1345	0.6282 ± 0.0154
ISOLET	0.4692 ± 0.0185	0.4921 ± 0.0268	0.5216 ± 0.0150	0.4827 ± 0.0404	0.5447 ± 0.0031	0.4270 ± 0.0000	0.4629 ± 0.0275	0.5320 ± 0.0022
LS	0.4572 ± 0.0856	0.4504 ± 0.2156	0.5274 ± 0.0073	0.5244 ± 0.1947	0.4632 ± 0.0152	0.5715 ± 0.0000	0.3090 ± 0.1029	0.6167 ± 0.0003
ODR	0.6410 ± 0.0608	0.7644 ± 0.0249	0.8262 ± 0.0511	0.7832 ± 0.0067	0.8404 ± 0.0070	0.6047 ± 0.0000	0.7560 ± 0.0062	0.9545 ± 0.0000
PD	0.5589 ± 0.0404	0.6492 ± 0.0399	0.6987 ± 0.0258	0.6105 ± 0.0247	0.6594 ± 0.0200	0.5298 ± 0.0000	0.6052 ± 0.0366	0.7971 ± 0.0040

, respectively.

As shown in Table 4, Table 5 and Table 6, the performance of ECM-RCL surpasses that of the other compared methods on most datasets, particularly on large-scale ones. This is because ECM-RCL accounts for structural differences among connective matrices in the label space and captures rich multi-scale structural information of samples in the feature space. K-means achieves the lowest clustering performance due to the lack of an ensemble learning strategy. In contrast to K-means, ECAR introduces a novel autoencoder with an attention mechanism that adaptively learns both sample representations and weights for base clustering results, thereby enhancing ensemble clustering performance. CEAM iteratively performs representation learning and structural optimization on a multiplex graph. This process enables mutual enhancement between the base clustering results and the ensemble clustering result, ultimately achieving better performance. LWEA improves the quality of the CA matrix by introducing a cluster weighting strategy, while ECPCS-HC further refines the CA matrix by considering indirect association relationships among clusters. Compared to other methods, EC-CMS and DREC achieve better performance on the datasets, primarily due to their more comprehensive optimization strategies and effective incorporation of high-confidence information.

Furthermore, according to the results in Table 4, Table 5 and Table 6, the proposed ECM-RCL yields relatively poor performance on the Lymphoma, WDBC, and Zoo datasets, performing comparably to, or even worse than, some ensemble clustering methods. The potential reasons for this are analyzed as follows: (1) These datasets have complex structures that contain redundant information, which weakens the accuracy of the symmetric error matrix of ECM-RCL. (2) These datasets may have a high degree of overlap between clusters, making it difficult to effectively align the multi-order graph structure information extracted from the feature space with the information in the label space. (3) The multi-scale structure of samples in the feature space introduces redundant information, which compromises the clarity of the consensus matrix. (4) Although the cluster structure and noise in the base clustering results within the label space are considered, other information in the base clustering results (e.g., redundant information) has not been fully utilized. Future study could explore adaptive model ensembles that integrate the dataset’s original features to address these limitations.

4.2.2. Statistical Analysis

In this subsection, we employ the Friedman test [42], the Wilcoxon signed-rank test [44], and the post hoc Holm test [45] to evaluate the differences in clustering performance (i.e., ACC) between ECM-RCL and all the adopted comparative methods on all the adopted datasets.

Table 7. The Friedman test of all the adopted methods.

Methods	Rank	$\mathit{p}$-Value	The Null Hypothesis
K-means	6.0789	0	rejected
LWEA	4.6579
DREC	3.7105
ECPCS-HC	4.8421
EC-CMS	4.6053
ECAR	5.5263
CEAM	5.2105
ECM-RCL	1.3684

presents the results of the Friedman test, where the null hypothesis assumes no significant differences among all the adopted methods. As shown in Table 7, the $p$-value is less than $0.05$, leading to the rejection of the null hypothesis, which indicates significant differences in the clustering performance of the adopted methods.

Table 8. The Wilcoxon signed-rank test of all the adopted methods.

Methods	$\mathit{p}$-Value	The Null Hypothesis
$\mathrm{K}\text{-means} \mathrm{vs}.$ ECM-RCL	0.000143	rejected
$\mathrm{LWEA} \mathrm{vs}.$ ECM-RCL	0.000168	rejected
$\mathrm{DREC} \mathrm{vs}.$ ECM-RCL	0.000271	rejected
$\text{ECPCS-HC} \mathrm{vs}.$ ECM-RCL	0.00058	rejected
$\text{EC-CMS} \mathrm{vs}.$ ECM-RCL	0.000121	rejected
$\mathrm{ECAR} \mathrm{vs}.$ ECM-RCL	0.000673	rejected
$\mathrm{CEAM} \mathrm{vs}.$ ECM-RCL	0.000121	rejected

presents the Wilcoxon signed-rank test results for pairwise comparisons between ECM-RCL and each comparative method, assuming no significant difference for each method pair. As shown in Table 8, all the $p$-values are less than $0.05$, and we therefore reject the null hypothesis, indicating significant differences in the clustering performance between ECM-RCL and each adopted comparative method.

Table 9. The post hoc Holm test of all the adopted methods.

$\mathit{i}$	Methods	$\mathit{z}=\frac{{\mathit{R}}_0-{\mathit{R}}_{\mathit{i}}}{\mathbf{SE}}$	$\mathit{p}$-Value	$\frac{\mathit{\alpha }}{\mathit{i}}$	The Null Hypothesis
7	K-means $\mathrm{vs}.$ ECM-RCL	5.927282	0	0.007143	rejected
6	ECAR $\mathrm{vs}.$ ECM-RCL	5.231903	0	0.008333	rejected
5	CEAM $\mathrm{vs}.$ ECM-RCL	4.834543	0.000001	0.01	rejected
4	ECPCS-HC $\mathrm{vs}.$ ECM-RCL	4.370957	0.000012	0.0125	rejected
3	LWEA $\mathrm{vs}.$ ECM-RCL	4.138164	0.000035	0.0167	rejected
2	EC-CMS $\mathrm{vs}.$ ECM-RCL	4.072937	0.000046	0.025	rejected
1	DREC $\mathrm{vs}.$ ECM-RCL	2.947084	0.003208	0.05	rejected

presents post hoc Holm test results for pairwise comparisons between ECM-RCL and each comparative method, assuming no significant difference for each method pair. According to [42], if $p\text{-value} < \alpha /i$, we can conclude that the two methods have significant differences, where $i$ represents the ranking of the $i$-th comparative method and $\alpha =0.05$. As shown in Table 9, all the $p$-values are less than $\alpha /i$, and we therefore reject the null hypothesis, indicating significant differences in the clustering performance between ECM-RCL and each adopted comparative method.

In summary, ECM-RCL exhibits significant differences compared with the other comparative methods. Moreover, based on the results in Table 4, Table 5 and Table 6, ECM-RCL demonstrates superior clustering performance over the adopted comparative methods.

4.2.3. Ablation Study

In this subsection, due to space limitations, we compare the proposed ECM-RCL with several of its degraded variants on the 9 adopted datasets (i.e., GLIMA, Heart, Iris, LM, Orlraws10p, WarpPIE10p, WDBC, Wine, Zoo) to validate the effectiveness and importance of each key component, which cover diverse scenarios ranging from low-dimensional to ultra-high-dimensional data, small-sample to medium-scale settings, and varying numbers of clusters.

EECM-RCL: It removes the noise matrices (i.e., $\left\{{\mathit{E}}_1,{\mathit{E}}_2,\dots ,{\mathit{E}}_m\right\}$) and directly recovers the consensus matrix from the connective matrices.

PECM-RCL: It removes the mapping models (i.e., $\left\{{\mathit{P}}_1,{\mathit{P}}_2,\dots ,{\mathit{P}}_m\right\}$) and directly performs weighted fusion of the denoised connective matrices.

RECM-RCL: It removes the theoretical rank constraint.

DECM-RCL: It refines the consensus matrix using first-order graph structure in the feature space.

As shown in

Table 10. Ablation study under three indices of ACC, NMI, and ARI.

	Methods	Indices	EECM-RCL	PECM-RCL	RECM-RCL	DECM-RCL	ECM-RCL
Datasets
GLIMA		ACC	0.6000 ± 0.0452	0.6080 ± 0.0551	0.6200 ± 0.0639	0.6320 ± 0.0567	0.6440 ± 0.0430
		NMI	0.5143 ± 0.0353	0.4986 ± 0.0380	0.5010 ± 0.0322	0.5171 ± 0.0183	0.5253 ± 0.0198
		ARI	0.3763 ± 0.0563	0.3652 ± 0.0611	0.3601 ± 0.0366	0.3817 ± 0.0300	0.3854 ± 0.0311
Heart		ACC	0.6200 ± 0.0058	0.6189 ± 0.0064	0.6163 ± 0.0043	0.6222 ± 0.0000	0.6244 ± 0.0053
		NMI	0.0367 ± 0.0053	0.0375 ± 0.0033	0.0378 ± 0.0184	0.0408 ± 0.0000	0.0409 ± 0.0042
		ARI	0.0539 ± 0.0059	0.0530 ± 0.0062	0.0505 ± 0.0041	0.0562 ± 0.0000	0.0584 ± 0.0052
Iris		ACC	0.9100 ± 0.0248	0.9080 ± 0.0208	0.9067 ± 0.0000	0.9067 ± 0.0000	0.9140 ± 0.0685
		NMI	0.8029 ± 0.0411	0.7985 ± 0.0305	0.7960 ± 0.0000	0.7960 ± 0.0000	0.8131 ± 0.0853
		ARI	0.7787 ± 0.1101	0.7641 ± 0.0494	0.7592 ± 0.0000	0.7592 ± 0.0000	0.7975 ± 0.1174
LM		ACC	0.5122 ± 0.0096	0.5008 ± 0.0166	0.5169 ± 0.0070	0.5058 ± 0.0161	0.5200 ± 0.0140
		NMI	0.6412 ± 0.0057	0.6337 ± 0.0099	0.6403 ± 0.0055	0.6398 ± 0.0026	0.6460 ± 0.0082
		ARI	0.3828 ± 0.0099	0.3705 ± 0.0162	0.3825 ± 0.0068	0.3697 ± 0.0180	0.3895 ± 0.0122
Orlraws10p		ACC	0.8470 ± 0.0432	0.8400 ± 0.0000	0.8430 ± 0.0067	0.8450 ± 0.0341	0.8470 ± 0.0408
		NMI	0.9113 ± 0.0310	0.9097 ± 0.0279	0.9172 ± 0.0159	0.9156 ± 0.0233	0.9207 ± 0.0158
		ARI	0.8161 ± 0.0106	0.8148 ± 0.0502	0.8285 ± 0.0222	0.8208 ± 0.0452	0.8330 ± 0.0218
WarpPIE10p		ACC	0.4714 ± 0.0151	0.4671 ± 0.0163	0.4648 ± 0.0121	0.4648 ± 0.0181	0.4762 ± 0.0135
		NMI	0.5528 ± 0.0093	0.5512 ± 0.0141	0.5540 ± 0.0042	0.5512 ± 0.0083	0.5562 ± 0.0127
		ARI	0.3132 ± 0.0159	0.3159 ± 0.0194	0.3121 ± 0.0112	0.3072 ± 0.0468	0.3181 ± 0.0129
WDBC		ACC	0.8896 ± 0.0190	0.8842 ± 0.0188	0.8989 ± 0.0124	0.8793 ± 0.0258	0.8903 ± 0.0191
		NMI	0.5026 ± 0.0420	0.4878 ± 0.0436	0.5218 ± 0.0481	0.4817 ± 0.0733	0.5045 ± 0.0441
		ARI	0.6028 ± 0.0613	0.5862 ± 0.0597	0.6329 ± 0.0405	0.5706 ± 0.0825	0.6050 ± 0.0615
Wine		ACC	0.7247 ± 0.0000	0.7247 ± 0.0000	0.7253 ± 0.0018	0.7247 ± 0.0000	0.7247 ± 0.0000
		NMI	0.4155 ± 0.0146	0.4231 ± 0.0138	0.4210 ± 0.0171	0.4217 ± 0.0119	0.4225 ± 0.0116
		ARI	0.4007 ± 0.0000	0.4007 ± 0.0000	0.4007 ± 0.0000	0.4007 ± 0.0000	0.4007 ± 0.0000
Zoo		ACC	0.7129 ± 0.0229	0.6960 ± 0.0959	0.7386 ± 0.0314	0.6366 ± 0.0247	0.7248 ± 0.0283
		NMI	0.7215 ± 0.0093	0.7015 ± 0.0201	0.7191 ± 0.0246	0.6217 ± 0.0137	0.7219 ± 0.0093
		ARI	0.6214 ± 0.0732	0.6061 ± 0.0322	0.6220 ± 0.0410	0.4636 ± 0.0124	0.6282 ± 0.0154

, ECM-RCL achieves the best clustering performance in most cases. In contrast, EECM-RCL suffers from performance degradation due to noise in the recovered consensus matrix, arising from ignoring noise in each connective matrix. PECM-RCL fails to account for differences in cluster structures across individual connective matrices, leading to structural disruption in the fused consensus matrix. RECM-RCL lacks modeling of the block-diagonal structure, which reduces the structural clarity of the consensus matrix. DECM-RCL ignores the multi-scale structural information of samples in the feature space, resulting in insufficient refinement of the consensus matrix and ultimately impairing clustering performance.

4.2.4. Parameter Sensitivity Analysis

In this subsection, we conduct a parameter sensitivity analysis with clustering accuracy (ACC) as the evaluation metric. Figure 2 presents the effects of parameters ${\lambda }_1$ and ${\lambda }_2$ on three adopted datasets (i.e., Heart, Iris, and LM). We can observe that the impact of parameter combinations on method performance varies among datasets. In general, ECM-RCL achieves favorable results when ${\lambda }_1\ge 0.1$ and ${\lambda }_2\ge 0.1$. Figure 3 presents the effect analysis of parameter ${\lambda }_3$ on three adopted datasets (i.e., Heart, Iris, and LM). The results indicate that the optimal value of ${\lambda }_3$ typically lies around $\left\{1,10\right\}$. The experimental results indicate that although different parameter combinations lead to fluctuations in the result, the overall magnitude of variation is limited. It is noteworthy that parameter ${\lambda }_3$ demonstrates significantly higher sensitivity compared to ${\lambda }_1$ and ${\lambda }_2$. A plausible explanation for this observation is that the quality of the mapping matrix directly influences the recovery of the consensus matrix. In addition,

Table 11. The comparison of the maximum order of ECM-RCL.

	ALLAML	GLIOMA	Heart	Iris	Lymphoma	Orlraws10p
2nd	0.7153 ± 0.0150	0.6440 ± 0.0430	0.6244 ± 0.0053	0.9140 ± 0.0685	0.6552 ± 0.0421	0.8470 ± 0.0408
3rd	0.7139 ± 0.0199	0.6440 ± 0.0741	0.6148 ± 0.0000	0.9220 ± 0.0340	0.6406 ± 0.0403	0.8560 ± 0.0425
4th	0.7139 ± 0.0360	0.6160 ± 0.0540	0.6111 ± 0.0000	0.9220 ± 0.0340	0.6365 ± 0.0356	0.8500 ± 0.0406
5th	0.7153 ± 0.0410	0.6240 ± 0.0523	0.6078 ± 0.0012	0.9213 ± 0.0345	0.6542 ± 0.0343	0.8500 ± 0.0406

reports the effect of the maximum order $V$ of the multi-order graph structures on clustering performance on six adopted datasets (i.e., ALLAML, GLIOMA, Heart, Iris, Lymphoma, and Orlraws10p). It is evident that the method generally performs better when $V=2$ or $V=3$.

Additionally, although the above parameter analysis results have confirmed that the parameter setting by grid search is reliable in our experiments, the grid search is difficult to implement in applications, and the settings of these parameters should be further improved. From the parameter analysis results, we can observe that ECM-RCL exhibits stable performance with the change in parameters ${\lambda }_1$ and ${\lambda }_2$, so their search ranges are appropriately precise to ${\lambda }_1\in \left\{0.1,1,10,100,1000\right\}$, ${\lambda }_2\in \left\{0.1,1,10,100,1000\right\}$ in the applications. For parameters ${\lambda }_3$ and $V$, they have the highest sensitivity in the performance of ECM-RCL, and they should be further precisely set according to the structure of datasets in the application, which has also been confirmed in [24].

4.2.5. Robustness Analysis of Base Clustering Results

In this subsection, the influence of both the quantity and variety of base clustering results on the ECM-RCL is investigated. The evaluation is conducted from two perspectives: (1) assessing the impact of the ensemble sizes $m$ on performance, and (2) comparing the results when using spectral clustering as an alternative to K-means for generating base clustering results. For spectral clustering, the number of base clustering results is set to 10 (i.e., $m=10$), and the number of clusters within each is randomly selected from $\left[c,\max \left(100,\sqrt{n}\right)\right]$. Experiments are performed on seven datasets (i.e., CLL_SUB, Heart, Iris, Lymphoma, Orlraws10p, Wine, Zoo). Figure 4 presents the average ACC, NMI, and ARI results of ECM-RCL under different $m$, while Figure 5 compares the average ACC, NMI, and ARI results obtained using K-means and spectral clustering.

As shown in Figure 4, although the performance of ECM-RCL varies slightly with different numbers of base clustering results, the experimental results demonstrate that the method achieves optimal performance when the number of base clustering results is set to 10 (i.e., $m=10$). This finding is consistent with the parameter setting in [40], further validating the rationality of this parameter selection. From Figure 5, it can be observed that ECM-RCL also exhibits stable clustering performance when K-means and spectral clustering are used as the base clustering methods.

In summary, ECM-RCL demonstrates good robustness and reliability under different base clustering configurations.

4.2.6. Visual Analysis

In this section, we present a visual analysis of the consensus matrix learned by ECM-RCL, as well as the graph structural weights corresponding to different orders in the feature space. Specifically, Figure A1 visualizes the consensus matrix, while Figure A2, Figure A3, Figure A4, Figure A5, Figure A6, Figure A7 and Figure A8 illustrate the distribution of graph structural weights across different orders. The corresponding figures can be found in the Appendix B and Appendix C (Figure A1, Figure A2, Figure A3, Figure A4, Figure A5, Figure A6, Figure A7 and Figure A8).

As shown in Figure A1, most of the learned consensus matrices exhibit a clear block-diagonal structure. However, we also observe that a few matrices lack a distinct block-diagonal pattern, which may be due to ambiguous sample relationships within the corresponding datasets. Furthermore, results presented in Table 4, Table 5, Table 6 and Table 10 demonstrate that the superior clustering performance of ECM-RCL is closely associated with the presence of a well-defined block-diagonal structure in the consensus matrix.

As shown in Figure A2, Figure A3, Figure A4, Figure A5, Figure A6, Figure A7 and Figure A8, despite variations across the fifteen datasets, some consistent trends in the weight distributions of graph structures emerge. When the maximum order is 2, the second-order graph structure is dominant. When the maximum order is 3, the weights are more evenly distributed across the first, second, and third orders. Notably, when the maximum order reaches 4 or higher, the weights do not increase with order. In addition, the parameter analysis in Table 11 further validates the choice of 2 or 3 as a reasonable maximum order.

4.2.7. Execution Time and Convergence Analysis

This subsection presents the execution times of all adopted methods and the convergence analysis of ECM-RCL.

Table 12. Average execution time (s) on all adopted datasets.

	Methods	LWEA	DREC	ECPCS-HC	EC-CMS	ECAR	CEAM	ECM-RCL
Datasets
ALLAML		0.0278	1.1962	0.0076	0.0437	5.3731	0.2442	0.0124
CLL_SUB		0.0023	0.0191	0.0019	0.0179	6.1449	0.2303	0.0317
GLIOMA		0.0018	0.0248	0.0014	0.0014	4.7770	0.1304	0.0111
Heart		0.0035	0.0264	0.0025	0.0381	4.6620	0.4803	0.1100
Iris		0.0025	0.0201	0.0018	0.0183	4.3012	0.3162	0.1263
LM		0.0055	0.0630	0.0052	0.0640	7.5051	0.7737	0.2029
Lymphoma		0.0030	0.0544	0.0016	0.0080	5.9408	0.2820	0.1428
MF		0.0293	0.1930	0.0942	6.1873	22.2389	9.7868	18.3646
Orlraws10P		0.0028	0.0427	0.0023	0.0094	7.7261	0.2722	0.0563
USPS		0.0278	0.3771	0.0770	4.9987	18.8030	9.6544	21.2204
Vertebral		0.0045	0.0321	0.0042	0.0539	5.0715	0.5352	0.1206
WarpPIE10p		0.0042	0.0534	0.0039	0.0247	7.0190	0.4849	0.0877
WDBC		0.0057	0.0238	0.0111	0.4950	6.3375	1.8485	0.4830
Wine		0.0024	0.0204	0.0019	0.0263	4.6208	0.3426	1.3352
Zoo		0.0019	0.0368	0.0016	0.0128	4.7620	0.2212	0.0541
ISOLET		0.2668	3.2181	2.0049	339.9318	316.5550	249.7655	1035.0218
LS		0.2283	1.7801	1.1972	132.5164	100.9128	143.3467	872.7249
ODR		0.1741	1.8478	0.8966	65.0205	95.9148	130.5054	448.3944
PD		0.5057	2.2972	4.1880	732.8491	316.2745	442.8964	3118.2037

reports the execution times for each method. As shown in Table 12, LWEA and ECPCS-HC exhibit faster execution times, primarily because these methods construct the consensus matrix using heuristic strategies with fewer optimization steps. In contrast, EC-CMS and DREC are more time-consuming: the former processes first-order high-confidence information among samples, while the latter integrates the global structure of the samples. CEAM exhibits a relatively long runtime due to dynamically adjusting intra-cluster connections based on cluster similarity during clustering. The computation of ECAR is relatively time-consuming, primarily due to the complexity of its pre-training and training phases. Although ECM-RCL is not the slowest method across all datasets, its overall execution time remains relatively long, particularly on large-scale datasets, which is mainly attributed to its time complexity being cubic with respect to the number of samples. Nevertheless, this method can effectively account for the noise within connective matrices, the structural differences among these matrices, and the rich sample structure information extracted from the feature space, resulting in superior clustering performance across multiple datasets, especially on large-scale ones. Although the increase in sample size significantly raises the computational cost, the results in Table 4, Table 5 and Table 6 indicate that the improvement in the accuracy and stability of clustering performance is worthwhile. Furthermore, optimized acceleration strategies (e.g., the Nyström method [36] or anchor-based method [37]) can be employed to effectively reduce the time complexity.

Furthermore, to verify the convergence of ECM-RCL, we visualize its convergence process on three datasets (i.e., Iris, WarpPIE10p, and Zoo). Figure 6 illustrates the evolution of the objective function, showing that ECM-RCL typically converges within approximately 50 iterations.

5. Conclusions

In this study, we propose a novel ensemble clustering method, termed ECM-RCL. First, a symmetric error matrix is introduced for each connective matrix to identify its noise components. Based on these matrices, a set of mapping models is designed to recover a reliable consensus matrix. Second, in the feature space, multi-order graph structures are constructed via random walks to capture high-order associations among samples, enabling multi-granularity refinement of the consensus matrix. Finally, a rank constraint incorporating a block-diagonal structure is imposed to ensure the structural clarity of the consensus matrix. Experimental results demonstrate that ECM-RCL achieves competitive clustering performance compared to all the adopted comparative methods.

Although the proposed ECM-RCL achieves good performance, some limitations still exist. First, the optimization process of ECM-RCL remains computationally expensive. Developing more efficient optimization techniques is crucial, such as the Nyström method or anchor-based methods. Second, this study mainly focuses on differences in cluster structures across connective matrices, without considering the cluster weights within them. Future work will explore the distribution of cluster weights in the label space. Third, we will develop an adaptive deep learning hybrid model. This framework will leverage advanced techniques (e.g., graph attention networks) to achieve intelligent redundancy reduction in a data-driven manner, while adaptively adjusting its learning strategy based on inherent dataset characteristics (e.g., scale and noise patterns). Ultimately, this leads to a more powerful and versatile clustering framework. Fourth, we will devise efficient hyperparameter optimization strategies to ascertain optimal configurations.