Improved Multi-View Graph Clustering with Global Graph Refinement

Highlights

What are the main findings?

• This study proposes a view-specific fusion network (VSFN) that extracts and integrates node attribute and structural information into view-specific representation through a global self-attention mechanism and self-supervised clustering strategy.
• A learnable attention-driven aggregation strategy and cross-view fusion module are adopted to merge view-specific representations for consensus clustering.

What is the implication of the main finding?

• The IMGCGGR method significantly outperforms existing state-of-the-art multi-view graph clustering techniques across various benchmark datasets.
• This approach effectively addresses the issue of insufficient structural extraction in multi-view data while capturing both local and global graph properties, making it suitable for multi-source graph-structured data, multi-sensor fusion and geographic information systems data.

Abstract

The goal of multi-view graph clustering (MVGC) for remote sensing data is to obtain a consistent partitioning by capturing complementary and consensus information across multiple views. However, numerous ambiguous background samples in multi-view remote sensing data increase structural heterogeneity while simultaneously hindering effective information extraction and fusion. Existing MVGC methods cannot selectively integrate and fully refine both graph structure and node attribute information for consensus representation learning. Furthermore, current methods tend to overlook distant nodes, thus failing to capture the global graph structure. To solve these issues, we propose a novel method called Improved Multi-View Graph Clustering with Global Graph Refinement (IMGCGGR). Specifically, we first design a view-specific fusion network (VSFN) to extract and integrate node attribute and structural information into view-specific representation for each view. VSFN not only utilizes a global self-attention mechanism to enhance the global properties of structural information but also constructs a clustering loss through a self-supervised strategy to guide the view-specific clustering distribution assignment. Moreover, to enhance the capability of view-specific representation, a learnable attention-driven aggregation strategy is introduced to flexibly fuse the attribute and structural feature. Then, we adopt a cross-view fusion module to adaptively merge multiple view-specific representations for generating the final consensus representation. Comprehensive experiments show that IMGCGGR achieves significant clustering performance improvements over baseline methods across various benchmark datasets.

1. Introduction

The advancement in multimedia data collection capabilities has significantly enhanced the availability of multi-view remote sensing data [1,2], which has been demonstrated to improve performance by comprehensively utilizing complementary knowledge from different data perspectives [3,4,5,6]. Take land remote sensing as an example: land data could be broken down into high-resolution optical images, SAR data, and night-time light data. High-resolution optical images can provide spectral information of the surface, the SAR data capture the structure and texture features of the surface, and the night-time light data can reflect the intensity of human activities. Regarding each modality as a data view, multi-view clustering [7,8] has recently attracted significant attention as crucial multimedia technology. By optimally fusing complementary and consensus information, it effectively reveals underlying data patterns. There exist complex relationships between different views of multi-view data. On the one hand, each view provides unique view-specific information. On the other hand, all views share the same semantic information. Effectively and fully integrating information from different views is an extremely challenging task in multi-view scenarios [9].

Although numerous studies have confirmed the power of multi-view clustering [10,11], traditional approaches often rely on shallow models with limited representation learning capabilities. These methods struggle to effectively address non-Euclidean structured data and comprehensively integrate graph topological information. Unlike common multi-view data, graph-structured multi-view data contain more abundant information, comprising both node attributes across multiple views and topological relationships. For instance, in the land use scenario, different areas on the surface are regarded as nodes in the diagram, and the spatial adjacency, functional correlation or similarity between the nodes constitute the connection relationships of the edges. Adjacent areas often have similar land types, while areas with functional connections (such as commercial areas and residential areas) may be spatially separated but functionally closely linked. These structural patterns, emerging from intrinsic data properties and neighborhood interactions, explicitly encode inter-sample dependencies. It is not only crucial for data representation learning but also essential for improving partitioning [12]. For clustering tasks in particular, fully leveraging the topological structure can greatly enhance clustering performance.

The emergence of the graph neural network (GNN) [13], as a powerful tool for analyzing non-Euclidean graph-structured data, has provided significant assistance in addressing a variety of graph analytical tasks across different domains, such as recommendation systems [14], information retrieval [15], and bioinformatics [16]. Inspired by the powerful representation extraction ability of GNNs, deep multi-view graph clustering (DMVGC) [17,18] algorithms are proposed to mine the graph-structured multi-view data. DMVGC methods typically integrate the graph structure into node attribute for multi-view representation learning. For example, Fan et al. [19] train a graph convolutional autoencoder to learn the shared representation of multi-view graph data by focusing on minimizing the reconstruction error of the graph view. Cheng et al. [20] design dual encoders that reconstruct the two sources embeddings in an attention mechanism with shared parameters.

Although the previous efforts have achieved significant performance improvements for graph-structured multi-view data, several limitations still exist. Existing deep multi-view graph clustering algorithms cannot flexibly integrate and fully refine both graph structure and node attribute information for multi-view representation learning. Information from attributes and structures is simply aligned or utilized with shared parameters, resulting in inadequate information interaction and merging. Moreover, these methods tend to leverage short-range adjacency connections, thus ignoring the global properties of structural information. In graph-structured data, nodes may exhibit implicit associations through multi-hop paths (e.g., functional similarity between distant proteins in interaction networks). Neglecting global structural information can lead to erroneous similarity assessments.

Motivated by the above observations, in this paper, we put forward a novel deep multi-view graph clustering framework for graph-structured multi-view data, which is called improved multi-view graph clustering with global graph refinement (IMGCGGR). IMGCGGR consists of two core modules: the view-specific fusion network (VSFN) module and the cross-view fusion module. The VSFN module is designed to fully extract and flexibly integrate view-specific node attributes and structural features by a graph autoencoder and self-supervised strategy. Specifically, the graph autoencoder not only enables a more comprehensive refinement of structural information through the global self-attention mechanism but also facilitates flexible integration with attribute features via a learnable attention-driven strategy. The self-supervised strategy is introduced to construct clustering loss for guiding the view-specific clustering distribution assignment. For the cross-view fusion module, it adaptively combines multiple view-specific representations to produce the final consensus representation. Our proposed method is suitable for processing and analyzing multi-source graph-structured and remote sensing data, including but not limited to multi-attribute graph data, multi-sensor data fusion, and geographic information system data with complex spatial relationships. To summarize, our major contributions are listed as follows:

• We propose a novel method called improved multi-view graph clustering with global self-attention (IMGCGGR). In the IMGCGGR, through the view-specific fusion network and cross-view fusion module, the node attribute and structural feature in graph-structured multi-view data can be thoroughly extracted and flexibly integrated. It greatly improves the clustering performance of graph-structured multi-view data.
• We introduce a global self-attention mechanism in the view-specific fusion network to enhance the global properties of structural information. Moreover, to enhance the reliability and capability of view-specific representation, a self-supervised strategy is designed to guide the view-specific clustering distribution assignment. By constructing these modules, the representation learning capability of IMGCGGR can be further improved.
• Extensive experiments on widely used benchmark graph-structured multi-view datasets demonstrate that our proposed IMGCGGR achieves significant improvements against existing state-of-the-art methods.

2. Related Work

In this section, we first present the basic principles of multi-view clustering and then introduce several classical multi-view clustering methods based on traditional machine learning. Furthermore, various recent studies in the literature for deep multi-view graph clustering are also discussed.

2.1. Basic Principles of Multi-View Clustering

In the multi-view scenario, the effectiveness of clustering is mainly ensured by two basic principles: complementary and consensus [7]. Of course, these two fundamental principles also serve as key guidelines for modeling multi-view clustering. The essence of the complementary principle lies in the idea that to provide a more comprehensive description of multi-view data objects, multiple views should be utilized more accurately. By leveraging complementary information contained in different views, a more thorough analysis of multi-view data can be achieved. The goal of the consensus principle is to maximize the shared consistency information among different views in order to reduce the probability of inconsistencies and errors. In conclusion, after years of investigation by scholars, it has been confirmed that the complementary and consensus principles have a crucial impact on the performance of multi-view clustering.

2.2. Traditional Multi-View Clustering

Traditional multi-view clustering (MvC) methods [21,22,23,24,25] enhance clustering performance by fusing features from multiple views through strategies such as subspace learning, graph fusion, or matrix factorization. We briefly introduce some classic methods in recent years. As one of the early pioneering works, Zhang et al. [26] propose a latent multi-view subspace clustering method, which reconstructs latent representations by leveraging complementary information in views, leading to more accurate and robust clustering results compared to single view. Luo et al. [27] design a multi-view subspace clustering (CSMSC) method that simultaneously utilizes both consistency and specificity for subspace representation learning. In CSMSC, consistency models the shared properties among all views, while specificity captures the intrinsic differences within each view. Wang et al. [28] take into account the issue of weights for different views and propose a general multi-view graph clustering algorithm. It derives a unified graph matrix by automatically weighting each view’s graph matrix and directly provides the final clustering results. To address the issue of noise and redundant information in the similarity graph learned from original multi-view features, Li et al. [29] propose a multi-view clustering method based on consensus graph learning (CGL). CGL constructs a similarity graph in the spectral embedding space and unifies spectral embedding with low-rank tensor learning into an overall optimization framework. Liu et al. [30] introduce a multi-view subspace clustering method that addresses the issue of redundant information in original views. By leveraging eigendecomposition to obtain low redundancy robust data, it lays the foundation for subsequent clustering. Inspired by the efficiency of anchor-based methods, Wen et al. [31] devise a novel anchor-based multi-view graph clustering framework to achieve global and local structure preservation. Considering the data-unpaired problem in the multi-view field, Wen et al. [32] propose a novel parameter-free unpaired multi-view graph clustering algorithm with cross-view structure matching (UPMGC). UPMGC adopts a unified framework to handle fully and partially unpaired multi-view data, effectively utilizing the structural information from each view to refine cross-view correspondences. Furthermore, to address the lack of flexibility and efficiency when constructing affinity matrices through tensors, Cai et al. [33] present an algorithm called tensorized scaled simplex representation (TSSR) for multi-view clustering. TSSR leverages a low-rank tensor constraint to capture consensus and complementary information among views while preserving the intrinsic relationships of the data. Owing to performing poorly on linearly non-separable data by a multi-view k-means algorithm, Lu et al. [34] propose an efficient multi-view k-means and use the Butterworth filters function to transform the adjacency matrix into a distance matrix. Considering the heterogeneity, high dimensionality, and label scarcity of remote sensing data, Cai et al. [35] propose an anchor-based multi-view kernel subspace clustering framework. This approach randomly samples anchors to construct sparse similarity matrices for computational efficiency and employs multi-kernel learning to automatically determine optimal kernel functions for different data modalities.

Although traditional methods have achieved certain effectiveness in multi-view data partitioning tasks [36,37,38], they share a common limitation: their tendency to capture shallow representations of multi-view data with restricted representation learning capacity. This is because most traditional methods employ shallow representation learning models (such as matrix factorization), and their consensus representations generated in the cross-view strategy cannot participate in the iterative optimization process of the algorithm. For graph-structured multi-view data in particular, even graph-based MvC algorithms cannot directly and sufficiently extract graph structural features. To address these limitations of traditional methods, in our proposed IMGCGGR, we adopt a global self-attention graph autoencoder for deep representation learning to fully extract the graph structure features. Meanwhile, a flexible cross-view fusion module is designed to generate the final trainable consensus representation to participate in iterative optimization process.

2.3. Deep Multi-View Graph Clustering

Deep multi-view graph clustering is an emerging research field, which automatically learns the nonlinear feature representation and graph structure through a graph neural network. Considering that shallow models’ ability to handle complex relationships in multi-view graphs can severely limit the modeling capacity of multi-view graph information, Fan et al. [19] are the first to attempt applying deep learning techniques to attributed multi-view graph clustering, which learns node embeddings and shared feature representations by utilizing single-view attribute feature and multiple graph structure views. Xia et al. [17] enhance node attributes by constructing a new Euler transform view while imposing a block diagonal representation constraint on the self-expression coefficient matrix measured by the $l_{1,2}$-norm. To address the issue of multi-view graph clustering being susceptible to low-quality graphs, Ling et al. [18] proposed a dual label-guided graph refinement method for multi-view graph clustering. They extract high-level view-common information to refine each view’s graph and reduce the influence of non-homophilous edges.

The aforementioned deep multi-view graph clustering methods mainly focus on multiple graphs, thereby ignoring the node attribute multi-view scenarios. To achieve the clustering of multiple attribute views, Cheng et al. [20] propose a novel multi-view attribute graph convolution (MAGCN) network and utilize two-pathway encoders to map graph embedding features. The first pathway reduces noise and redundancy using multi-view attribute graph attention networks, and the second pathway employs consistent embedding encoders to model geometric relationships. Xia et al. [39] propose a self-supervised graph convolutional network for multi-view clustering (SGCMC) to optimize node content reconstruction and graph structure reconstruction with weighting sharing. It jointly optimizes latent representations and coefficient matrices using clustering labels, enabling an iterative refinement of node clustering. Wang et al. [40] introduce a consistent multiple graph embedding clustering framework (CMGEC) and devise a mutual information maximization module to guide the common representation. To flexibly encode the complementary information of multi-view data, it uses a multi-graph attention fusion encoder. Xiao et al. [41] propose an efficient model called the dual fusion-propagation graph neural network to capture multiple pieces of information among different views. This module’s view-specific propagation effectively learns discriminative view representations while preserving complementarity.

Despite significant progress, existing deep multi-view graph clustering methods still face challenges in effectively combining node and graph representations. Most existing methods use standard graph convolutional networks for representation learning. They extract local features by aggregating neighbor node information, which leads to insufficient structural information capture. During the fusion mechanism, when integrating to common representation or a similarity graph, these methods typically employ a simplistic weighted-sum technique for construction. It may fail to fully explore the implicit inter-view relationships. In addition, most existing models employ the local attention mechanism and emphasize local structural information while neglecting global properties, which impacts their subsequent fusion capability. To overcome the aforementioned shortcomings, our proposed IMGCGGR introduces a global self-attention mechanism for comprehensive graph representation learning. By jointly training it with weighted local graph learning, we achieve more sufficient structural information extraction. Meanwhile, when integrating the structural information and attribute representations, unlike the simplistic weighted-sum techniques used in existing deep multi-view graph clustering methods, our approach fuses structural and attribute information through an attention-driven weighting strategy during joint training.

3. Methods

In this section, we comprehensively elaborate the proposed IMGCGGR method. IMGCGGR contains two fundamental modules: the view-specific fusion network (VSFN) module and the cross-view fusion module. The overall framework illustration is shown in Figure 1. The VSFN first extracts structural representation ${\mathbf{S}}^{\left(v\right)}$ and attributed representation ${\mathbf{H}}^{\left(v\right)}$ through a view-specific global attention module, and then it transforms both into probability distributions using Student’s t-distribution. After combining structural and attribute representation into the fused representation ${\mathbf{Z}}^{\left(v\right)}$, the training of the clustering distribution is guided by a view-specific self-supervised module. The cross-view fusion module learns to weight and combine view-specific fused representations through a trainable framework, yielding an improved consensus representation.

Table 1. The main notations and descriptions.

Notation	Description
${\mathbf{X}}^{\left(v\right)}$$\in {\mathbb{R}}^{d_v\times N}$	attribute matrix
${\mathbf{A}}^{\left(v\right)}\in {\mathbb{R}}^{N\times N}$	undirected original adjacency matrix
${\tilde{\mathbf{A}}}^{\left(v\right)}\in {\mathbb{R}}^{N\times N}$	renormalized adjacency matrix
${\widehat{\mathbf{X}}}^{\left(v\right)}\in {\mathbb{R}}^{d_v\times N}$	reconstructed attribute matrix
${\widehat{\mathbf{A}}}^{\left(v\right)}\in {\mathbb{R}}^{N\times N}$	reconstructed adjacency matrix
${\mathbf{H}}^{\left(v\right)}\in {\mathbb{R}}^{d_{(v,l)}\times N}$	view-specific attributed feature
${\mathbf{S}}^{\left(v\right)}\in {\mathbb{R}}^{d_{(v,l)}\times N}$	view-specific structural feature
${\mathbf{Z}}^{\left(v\right)}\in {\mathbb{R}}^{d_{(v,l)}\times N}$	fused view-specific representation
${\mathbf{Q}}_{\mathit{h}}^{\left(v\right)}\in {\mathbb{R}}^{K\times N}$	soft assignment distribution of ${\mathbf{H}}^{\left(v\right)}$
${\mathbf{Q}}_{\mathit{s}}^{\left(v\right)}\in {\mathbb{R}}^{K\times N}$	soft assignment distribution of ${\mathbf{S}}^{\left(v\right)}$
$\mathbf{Z}\in {\mathbb{R}}^{K\times N}$	final consensus representation

presents the main notations used in this paper. In the following, we will provide detailed implementation descriptions of each module.

3.1. Notations and Problem Definition

Let there be a multi-view undirected graph $\mathcal{G}=\{\mathcal{V},\mathcal{E}\}$ with K clusters, where $\mathcal{V}=\{v_1,\dots ,v_N\}$ and $\mathcal{E}$ represent the node set and the edge set, respectively. The graph $\mathcal{G}$ is typically characterized by attribute matrix $\mathbf{X}=\{{\mathbf{X}}^{\left(1\right)},{\mathbf{X}}^{\left(2\right)},\dots ,{\mathbf{X}}^{\left(v\right)},\dots \}$ and undirected original adjacency matrix $\mathbf{A}=\{{\mathbf{A}}^{\left(1\right)},{\mathbf{A}}^{\left(2\right)},\dots ,{\mathbf{A}}^{\left(v\right)},\dots \}$. ${\mathbf{X}}^{\left(v\right)}\in {\mathbb{R}}^{d_v\times N}$ represents the node attribute view, and ${\mathbf{A}}^{\left(v\right)}\in {\mathbb{R}}^{N\times N}$ represents the structural view. To mitigate the issue of gradient vanishing/exploding, we employ a renormalization trick:

$${\tilde{\mathbf{A}}}^{\left(v\right)}={\tilde{\mathbf{D}}}^{{\left(v\right)}^{-{\displaystyle \frac{1}{2}}}}({\mathbf{A}}^{\left(v\right)}+{\mathbf{I}}_N){\tilde{\mathbf{D}}}^{{\left(v\right)}^{-{\displaystyle \frac{1}{2}}}},$$

(1)

where ${\tilde{\mathbf{D}}}_{ii}^{\left(v\right)}={\sum }_j{({\mathbf{A}}^{\left(v\right)}+{\mathbf{I}}_N)}_{ij}$ is the degree matrix, and ${\mathbf{I}}_N$ denotes the identity matrix. N is the number of nodes. When performing calculations using the graph structure later on, it is assumed that the preprocessed $\tilde{\mathbf{A}}=\{{\tilde{\mathbf{A}}}^{\left(1\right)},{\tilde{\mathbf{A}}}^{\left(2\right)},\dots ,{\tilde{\mathbf{A}}}^{\left(v\right)},\dots \}$ is used for the computations by default.

3.2. View-Specific Fusion Network

In order to fully extract the node attributes and structural information representations from graph-structured multi-view data, and to flexibly integrate both of them into the specific representations for each view, we introduce a view-specific fusion network. The flowchart of the view-specific fusion network is shown in Figure 2. The view-specific fusion network module consists of two submodules: a view-specific global self-attention module and a view-specific self-supervised module. In general, the view-specific global self-attention module is designed to refine the view-specific structural representation ${\mathbf{S}}^{\left(v\right)}$. Then, the learned structural representation ${\mathbf{S}}^{\left(v\right)}$ is fused with the attributed representation ${\mathbf{H}}^{\left(v\right)}$ by a learnable attention-driven strategy to generate the fused view-specific representation ${\mathbf{Z}}^{\left(v\right)}$. Here, the attributed representation ${\mathbf{H}}^{\left(v\right)}$ is generated by mapping the attribute matrix ${\mathbf{X}}^{\left(v\right)}$ into low-dimensional embedding through an autoencoder network. Finally, the fused view-specific representation ${\mathbf{Z}}^{\left(v\right)}$ is trained by a self-supervised module to achieve more reliable and clustering-friendly embedding.

3.2.1. View-Specific Global Self-Attention Module

Most existing deep multi-view graph clustering methods typically utilize a multi-layer graph convolution network (GCN) for graph representation learning. However, the standard GCN can only capture the local structure and cannot extract global structural information. Thus, we construct a view-specific global self-attention (VSGSA) module to address this limitation.

In the VSGSA module, we first transform the l-th hidden embedding ${\mathbf{U}}^{(v,l)}\in {\mathbb{R}}^{d_{(v,l)}\times N}$ (${\mathbf{U}}^{(v,0)}={\mathbf{X}}^{\left(v\right)}$) into two feature spaces $\mathit{f}$, $\mathit{g}$. The computation of the attention for ${\mathbf{U}}^{(v,l)}$ is formulated as follows:

$$\begin{array}{cc}\hfill & s_{ij}^{(v,l)}=\mathit{f}{({\mathbf{U}}_i^{(v,l)})}^T\mathit{g}({\mathbf{U}}_j^{(v,l)}),\hfill \\ \hfill & {\beta }_{i,j}^{(v,l)}={\displaystyle \frac{exp(s_{ij}^{(v,l)})}{{\sum }_{i=1}^{N}exp(s_{ij}^{(v,l)})}},\hfill \\ \hfill & \mathit{f}({\mathbf{U}}^{(v,l)})={\mathbf{W}}_{\mathit{f}}^{(v,l)}{\mathbf{U}}^{(v,l)},\mathit{g}({\mathbf{U}}^{(v,l)})={\mathbf{W}}_{\mathit{g}}^{(v,l)}{\mathbf{U}}^{(v,l)},\hfill \end{array}$$

(2)

${\mathbf{W}}_{\mathbf{f}}^{(v,l)},{\mathbf{W}}_{\mathbf{g}}^{(v,l)}$ are the trainable weight matrices. ${\beta }_{i,j}^{(v,l)}$ denotes the attention influence of node j on node i. $d_{(v,l)}$ is the dimension of the l-th layer network for the v-th view, and N is the number of samples. Thus, the output ${\mathit{o}}^{(v,l)}=\left({\mathit{o}}_{\mathbf{1}},{\mathit{o}}_{\mathbf{2}},\dots ,{\mathit{o}}_{\mathit{j}},\dots ,{\mathit{o}}_{\mathit{N}}\right)\in {\mathbb{R}}^{d_{(v,l)}\times N}$ of the global self-attention network is formally calculated as

$$\begin{array}{cc}\hfill & {\mathit{o}}_{\mathit{j}}=\mathit{v}(\sum _{i=1}^N{\beta }_{i,j}^{(v,l)}\mathit{h}({\mathbf{U}}_i^{(v,l)}))\hfill \\ \hfill & \mathit{h}({\mathbf{U}}_i^{(v,l)})={\mathbf{W}}_{\mathit{h}}^{(v,l)}{\mathbf{U}}_i^{(v,l)},\mathit{v}({\mathbf{U}}_i^{(v,l)})={\mathbf{W}}_{\mathit{v}}^{(v,l)}{\mathbf{U}}_i^{(v,l)}\hfill \end{array}$$

(3)

Here, ${\mathbf{W}}_{\mathit{h}}^{(v,l)}$ and ${\mathbf{W}}_{\mathit{v}}^{(v,l)}$ are the learned weight matrices. Then, we further integrate the global self-attention network with a standard GCN through linear weighting, aiming to simultaneously leverage both the global and local structure. The final learned view-specific structural representation ${\mathbf{S}}^{\left(v\right)}$ of the VSGSA module is formulated as follows:

$$\begin{array}{c}\hfill {\mathbf{S}}^{\left(v\right)}=\alpha \mathbf{GCN}({\mathbf{X}}^{\left(\mathit{v}\right)},{\tilde{\mathbf{A}}}^{\left(\mathit{v}\right)};{\mathsf{\Theta }}^{\left(v\right)})+\beta {\mathit{o}}^{\left(\mathit{v}\right)},\end{array}$$

(4)

where ${\mathsf{\Theta }}^{\left(v\right)}$ represents the set of learning parameter matrices of the GCN network. $\alpha$ and $\beta$ are the learnable scalar, and $\alpha$ is initialized as 1, while $\beta$ is initialized as 0. The reason for this initialization is that the learning of the local structure representation is relatively easy and efficient, because the local structure only needs to focus on the interaction operations with local neighbors. In contrast, the learning of global structure representation is relatively difficult, as it requires calculating the correlation with global nodes. Specifically, introducing a learnable scalar allows the network to initially rely on cues from local neighborhoods and then gradually learn to allocate more weight to global evidence.

To enhance representation learning, we integrate the structural representation ${\mathbf{S}}^{\left(v\right)}$ with the attributed representation ${\mathbf{H}}^{\left(v\right)}$ through an attention-driven aggregation strategy to generate the fused view-specific representation ${\mathbf{Z}}^{\left(v\right)}$. Specifically, we first concatenate the ${\mathbf{H}}^{\left(v\right)}$ and ${\mathbf{S}}^{\left(v\right)}$ to form a new concatenated representation ${\tilde{\mathbf{S}}}^{\left(v\right)}$:

$${\tilde{\mathbf{S}}}^{\left(v\right)}=[{\mathbf{H}}^{\left(v\right)}\left|\right|{\mathbf{S}}^{\left(v\right)}],$$

(5)

Then, we apply a LeakyReLU activation function (negative input slope is set as 0.2) to the product of ${\tilde{\mathbf{S}}}^{\left(v\right)}$ and the trainable parameter matrix ${\tilde{\mathbf{W}}}^{\left(v\right)}$. Furthermore, through softmax and $L_2$ regularization, we can compute the weight matrix ${\mathbf{M}}^{\left(v\right)}$ as

$${\mathbf{M}}^{\left(v\right)}=L_2(softmax(\sigma ({\tilde{\mathbf{S}}}^{\left(v\right)}{\tilde{\mathbf{W}}}^{\left(v\right)}))),$$

(6)

where $\sigma$ indicates the activation function and $L_2$ denotes the $l_2$-norm regularization. Here, softmax first performs global scaling, normalizing it into a probability distribution. The subsequent $L_2$ regularization quantifies the concentration degree of the distribution. The advantage of this design lies in that it not only ensures that the output weights are probabilities but also enables monitoring whether the attention in the representation learning has deteriorated (such as all zeros or a single peak). Thus, the fused view-specific representation ${\mathbf{Z}}^{\left(v\right)}$ is given by

$${\mathbf{Z}}^{\left(v\right)}={\mathbf{M}}_1^{\left(v\right)}\odot {\mathbf{H}}^{\left(v\right)}+{\mathbf{M}}_2^{\left(v\right)}\odot {\mathbf{S}}^{\left(v\right)},$$

(7)

where ${\mathbf{M}}_i^{\left(v\right)}$ represents an expanded matrix of the i-th dimension of ${\mathbf{M}}^{\left(v\right)}$. To train the VSGSA module, we need to simultaneously minimize the reconstruction loss of both the attribute representation and the structural representation:

$$\begin{array}{cc}\hfill & {\mathcal{L}}_{re}=\sum _{v=1}^M({∥{\mathbf{X}}^{\left(v\right)}-{\widehat{\mathbf{X}}}^{\left(v\right)}∥}_F^2+CELoss({\tilde{\mathbf{A}}}^{\left(v\right)},{\widehat{\mathbf{A}}}^{\left(v\right)})),\hfill \\ \hfill & CELoss({\tilde{\mathbf{A}}}^{\left(v\right)},{\widehat{\mathbf{A}}}^{\left(v\right)})=-{\displaystyle \frac{1}{N^2}}\sum _{i=1}^N\sum _{i=j}^N[{\tilde{\mathbf{a}}}_{ij}^{\left(v\right)}log({\widehat{\mathbf{a}}}_{ij}^{\left(v\right)})+(1-{\tilde{\mathbf{a}}}_{ij}^{\left(v\right)})log(1-{\widehat{\mathbf{a}}}_{ij}^{\left(v\right)})],\hfill \end{array}$$

(8)

where ${\tilde{\mathbf{a}}}_{ij}^{\left(v\right)}$ and ${\widehat{\mathbf{a}}}_{ij}^{\left(v\right)}$ are, respectively, the elements of ${\tilde{\mathbf{A}}}^{\left(v\right)}$ and ${\widehat{\mathbf{A}}}^{\left(v\right)}$. ${\widehat{\mathbf{X}}}^{\left(v\right)}$ is the reconstruction attributed view, which is generated by an autoencoder network. ${\widehat{\mathbf{A}}}^{\left(v\right)}$ is the reconstruction graph topological structure, which is implemented by ${\widehat{\mathbf{A}}}^{\left(v\right)}=\mathbf{sigmoid}({\mathbf{S}}^{\left(v\right)}{({\mathbf{S}}^{\left(v\right)})}^T)$. M represents the number of views.

Consider the theory of deep clustering, namely that the representations trained solely with reconstruction loss is not necessarily suitable for clustering tasks [42]. Furthermore, most existing multi-view clustering algorithms typically first learn a common representation and then apply k-means to obtain the final clustering assignment. This two-stage learning approach fails to leverage the clustering process to improve the generated representation. Therefore, we introduce a view-specific self-supervised module to guide the clustering distribution assignment.

3.2.2. View-Specific Self-Supervised Module

The proposed view-specific self-supervised module is constructed to guide a clustering-friendly fused representation. To achieve this goal, we introduce two clustering layers and devise the corresponding clustering loss in a self-supervised manner. Specifically, Student’s t-distribution is firstly adopted to respectively generate the soft assignments ${\mathbf{Q}}_{\mathit{h}}^{\left(v\right)}$(from ${\mathbf{H}}^{\left(v\right)}$) and ${\mathbf{Q}}_{\mathit{s}}^{\left(v\right)}$(from ${\mathbf{S}}^{\left(v\right)}$):

$$\begin{array}{c}\hfill q_{h_{ij}}^{\left(v\right)}={\displaystyle \frac{{\left(1+{∥{\mathbf{H}}_i^{\left(v\right)}-{\mu }_j^{\left(v\right)}∥}^2/\gamma \right)}^{-\frac{\gamma +1}{2}}}{{\sum }_{j^{\prime }}{\left(1+{∥{\mathbf{H}}_i^{\left(v\right)}-{\mu }_{j^{\prime }}^{\left(v\right)}∥}^2/\gamma \right)}^{-\frac{\gamma +1}{2}}}},\\ \hfill q_{s_{ij}}^{\left(v\right)}={\displaystyle \frac{{\left(1+{∥{\mathbf{S}}_i^{\left(v\right)}-{\upsilon }_j^{\left(v\right)}∥}^2/\gamma \right)}^{-\frac{\gamma +1}{2}}}{{\sum }_{j^{\prime }}{\left(1+{∥{\mathbf{S}}_i^{\left(v\right)}-{\upsilon }_{j^{\prime }}^{\left(v\right)}∥}^2/\gamma \right)}^{-\frac{\gamma +1}{2}}}},\end{array}$$

(9)

where $\gamma$ indicates the freedom degree of Student’s t-distribution. Here, the default value of the degree of freedom $\gamma$ is set to 1. This is because cross-validation cannot be performed in an unsupervised environment, and this parameter learning is unnecessary [43]. ${\mu }_j^{\left(v\right)},{\upsilon }_j^{\left(v\right)}$ are the cluster centroids initialized in the pre-training stage and updated during the fine-tuning stage. $q_{h_{ij}}^{\left(v\right)}$ and $q_{s_{ij}}^{\left(v\right)}$ denote the probability of assigning node i to cluster j.

Then, we respectively construct the target distributions ${\mathbf{P}}_{\mathit{h}}^{\left(v\right)}$ and ${\mathbf{P}}_{\mathit{s}}^{\left(v\right)}$ as follows:

$$\begin{array}{cc}\hfill p_{h_{ij}}^{\left(v\right)}& ={\displaystyle \frac{{(q_{h_{ij}}^{\left(v\right)})}^2/{\sum }_i{q}_{h_{ij}}^{\left(v\right)}}{{\sum }_{j^{\prime }}{(q_{h_{i{j}^{\prime }}}^{\left(v\right)})}^2/{\sum }_i{q}_{h_{i{j}^{\prime }}}^{\left(v\right)}}},\hfill \\ \hfill p_{s_{ij}}^{\left(v\right)}& ={\displaystyle \frac{{(q_{s_{ij}}^{\left(v\right)})}^2/{\sum }_i{q}_{s_{ij}}^{\left(v\right)}}{{\sum }_{j^{\prime }}{(q_{s_{i{j}^{\prime }}}^{\left(v\right)})}^2/{\sum }_i{q}_{s_{i{j}^{\prime }}}^{\left(v\right)}}},\hfill \end{array}$$

(10)

The target distributions ${\mathbf{P}}_{\mathit{h}}^{\left(v\right)}$ and ${\mathbf{P}}_{\mathit{s}}^{\left(v\right)}$ are introduced to refine the soft assignments through squared normalization, transforming the initially unreliable soft clustering distribution into a stable and highly discriminative target distribution. Subsequently, these two clustering layers can be used to guide the fused representation ${\mathbf{Z}}^{\left(v\right)}$. In particular, we first utilize the target distributions to guide their respective soft assignment distributions as follows:

$$\begin{array}{cc}\hfill & {\mathcal{L}}_{self}=\sum _{v=1}^M\{\mathrm{KL}({\mathbf{P}}_{\mathit{h}}^{\left(v\right)}\parallel {\mathbf{Q}}_{\mathit{h}}^{\left(v\right)})+\mathrm{KL}({\mathbf{P}}_{\mathit{s}}^{\left(v\right)}\parallel {\mathbf{Q}}_{\mathit{s}}^{\left(v\right)})\}\hfill \\ \hfill & =\sum _{v=1}^M(\sum _{i=1}^N\sum _{j=1}^K{p}_{h_{ij}}^{\left(v\right)}log{\displaystyle \frac{p_{h_{ij}}^{\left(v\right)}}{q_{h_{ij}}^{\left(v\right)}}}+\sum _{i=1}^N\sum _{j=1}^K{p}_{s_{ij}}^{\left(v\right)}log{\displaystyle \frac{p_{s_{ij}}^{\left(v\right)}}{q_{s_{ij}}^{\left(v\right)}}}),\hfill \end{array}$$

(11)

where K indicates the number of clusters.

Next, we exploit the target distributions ${\mathbf{P}}_{\mathit{s}}^{\left(v\right)}$ to guide the fused view-specific representation ${\mathbf{Z}}^{\left(v\right)}$ and establish the connectivity between ${\mathbf{H}}^{\left(v\right)}$ and ${\mathbf{S}}^{\left(v\right)}$ through the KL divergence:

$$\begin{array}{cc}\hfill {\mathcal{L}}_{guide}& =\sum _{v=1}^M\{\mathrm{KL}({\mathbf{P}}_{\mathit{s}}^{\left(v\right)}\parallel {\mathbf{Z}}^{\left(\mathit{v}\right)})+\mathrm{KL}({\mathbf{P}}_{\mathit{h}}^{\left(v\right)}\parallel {\mathbf{Q}}_{\mathit{s}}^{\left(v\right)})\},\hfill \end{array}$$

(12)

To simultaneously learn the fused view-specific representations and cluster assignments in a unified process, we can jointly train the reconstruction loss and clustering loss by minimizing the overall objective function:

$$L={\mathcal{L}}_{re}+{\lambda }_1{\mathcal{L}}_{self}+{\lambda }_2{\mathcal{L}}_{guide},$$

(13)

where ${\lambda }_1,{\lambda }_2>0$ are the trade-off parameters.

3.3. Cross-View Fusion Module

In order to fuse the view-specific representations learned from the view-specific fusion network module, we devise the cross-view fusion module. Unlike traditional methods where the consensus representation cannot participate in training, the cross-view fusion module is capable of adaptively integrating multiple view-specific representations and generating the final trainable consensus representation:

$$\mathbf{Z}=\sum _{v=1}^M{w}_v{\mathbf{Z}}^{\left(v\right)},w_v={\displaystyle \frac{e^{w_v}}{{\sum }_{i=1}^M{e}^{w_i}}},$$

(14)

Here, $w_v$ is a learnable variable that automatically adjusts the importance of the v-th preliminary representation and is updated through gradient descent optimization. Benefiting from a series of fusion mechanisms, the proposed method can flexibly integrate information from all input views. The final clustering probability distribution z is computed by the softmax function $z=softmax\left(\mathbf{Z}\right)$. The detailed training process of IMGCGGR is shown in Algorithm 1.

Algorithm 1 Multi-View Graph Clustering with Global Self-Attention

Input: attribute feature ${\left\{{\mathbf{X}}^{\left(v\right)}\right\}}_{v=1}^M$; adjacency matrix ${\left\{{\mathbf{A}}^{\left(v\right)}\right\}}_{v=1}^M$; cluster number K; iteration number $I_{\max }$. Output: Clustering result z 1: Initialize the parameters of autoencoder and global self-attention module to obtain ${\mathbf{H}}^{\left(v\right)}$ and ${\mathbf{S}}^{\left(v\right)}$; 2: Initialize the clustering centroids $\mu$, $\upsilon$; 3: while $iter\le I_{\max }$ do 4:      Update ${\mathbf{M}}^{\left(v\right)}$ and ${\mathbf{Z}}^{\left(v\right)}$ by Equations (6) and (7); 5:      Calculate the soft assignment distribution ${\mathbf{Q}}_{\mathit{h}}^{\left(v\right)}$ and ${\mathbf{Q}}_{\mathit{s}}^{\left(v\right)}$ by Equation (9); 6:      Compute the distribution ${\mathbf{P}}_{\mathit{h}}^{\left(v\right)}$ and ${\mathbf{P}}_{\mathit{s}}^{\left(v\right)}$ via Equation (10); 7:      Calculate ${\mathcal{L}}_{re}$, ${\mathcal{L}}_{self}$, and ${\mathcal{L}}_{guide}$ by Equation (8), Equation (11), and Equation (12), respectively; 8:      Update $\mathbf{Z}$ via Equation (14); 9:      Update the whole network by minimizing Equation (13); 10:    $iter=iter+1$; 11: end while 12: Obtain clustering result z; 13: return z

3.4. Complexity Analysis

The computational complexity of IMGCGGR consists of two parts: a view-specific fusion network (VSFN) and a cross-view fusion module. For a VSFN, updating the autoencoder costs $\mathcal{O}\left(N{\sum }_{v=1}^M{\prod }_{i=1}^l{d}_v{d}_{v,i}\right)$, where N denotes the number of samples, M represents the number of views, $d_v$ is the dimension of the v-th attribute view, and $d_{v,i}$ is the number of dimensions for the i-th layer. Calculating the global self-attention module costs $\mathcal{O}\left(\right(\left|\mathcal{E}\right|+N){\sum }_{v=1}^M{\prod }_{i=1}^l{d}_v{d}_{v,i})+M{N}^2)$, as the operation can be computed efficiently using sparse matrix computation. $\left|\mathcal{E}\right|$ denotes the number of edges. Performing a self-supervised module needs $\mathcal{O}(MNK+MNlogN)$. K is the cluster number. For the cross-view fusion module, it costs $\mathcal{O}\left(MNK\right)$.

4. Results

This section provides a detailed description of the experiments to verify the effectiveness of IMGCGGR, including the experimental setup, performance comparison, ablation studies, hyper-parameter analysis, running time, convergence analysis, and visualization analysis.

4.1. Experimental Setup

Datasets: To evaluate the effectiveness of IMGCGGR, we conduct experiments on six widely used benchmark datasets [44], including ACM, Amazon Photos (AMP), Citeseer, Cora, DBLP, and Pubmed. To enhance representation learning, we construct an additional attribute view through the Cartesian product according to [20].

ACM is a paper network from the ACM database, where edges connect the papers belonging to the same author. AMP is an co-purchase graph network from the Amazon database, where nodes represent goods, while edges indicate that two goods are frequently bought together. Citeseer, Cora, and Pubmed are a citation network dataset about scientific publications, where edges represent citations between publications. DBLP is an author network from the DBLP database, where edges represent the co-author pairs. The statistics of these datasets are summarized in

Table 2. Statistics of the datasets.

Datasets	Nodes	Dimensions		Classes	Edges
ACM	3025	1870	3025	3	13,128
AMP	7487	745	7487	8	119,043
Citeseer	3327	3703	3327	6	4614
Cora	2708	1433	2708	7	5278
DBLP	4057	334	4057	4	3528
Pubmed	19,717	500	19,717	3	44,326

.

Baseline Methods: For a comprehensive evaluation, we compare IMGCGGR with eleven representative state-of-the-art multi-view clustering methods, including GMC [28], CGL [29], CoMSC [30], EMVGC [31], UPGMC [32], UPCoMSC [32], TSSR [33], EMKIC [34], MAGCN [20], SGCMC [39], and GMGEC [40]. A brief introduction to these methods is provided below.

• GMC [28]: It is a representative graph-based MvC algorithm, which derives a unified graph matrix by automatically weighting the graph matrices of each view.
• CGL [29]: It constructs a similarity graph based on consensus graph learning in the spectral embedding space and unifies spectral embedding with low-rank tensor learning into an overall optimization framework.
• CoMSC [30]: It is a representative subspace-based MvC approach, which leverages eigendecomposition to obtain low-redundancy robust data.
• EMVGC [31]: It devises a novel anchor-based multi-view graph clustering framework to achieve global and local structure preservation.
• UPGMC and UPCoMSC [32]: It adopts a unified framework to handle fully and partially unpaired multi-view data, effectively utilizing the structural information from each view to refine cross-view correspondences.
• TSSR [33]: It leverages a low-rank tensor constraint to capture consensus and complementary information among views while preserving the intrinsic relationships of the data.
• EMKIC [34]: It uses the Butterworth filters function to transform the adjacency matrix into a distance matrix.
• MAGCN [20]: It is a representative deep multi-view graph clustering method designed to address the clustering of multi-view graph data, which utilizes two-pathway encoders to map graph embedding features and learn view-consistency information.
• SGCMC [39]: It exploits a self-supervised multi-view graph attention autoencoder to optimize node content reconstruction loss and graph structure reconstruction loss with weighting sharing.
• GMGEC [40]: It devises a graph autoencoder and introduces a multi-view mutual information maximization module to guide the learned common representation.

Implementation Details: In this paper, we employ four commonly used clustering metrics [39] to evaluate the clustering performance: accuracy (ACC), normalized mutual information (NMI), adjusted rand index (ARI), and macro F1 score (F1). For each compared baseline method, we compute the average and standard deviation results from ten independent runs under their recommended parameter configurations. For our proposed IMGCGGR, to obtain reliable clustering and avoid collapse, we first pre-train the autoencoder and view-specific global self-attention module, respectively, and then combine the self-supervised clustering loss to jointly train our model. The dimensions of each view-specific module encoder are set to 500–500–2000–20 for all benchmark datasets.

4.2. Performance Comparison

Table 3. The clustering performance comparison with traditional multi-view clustering baseline methods on six benchmark datasets (mean% ± std%). The best results are highlighted in bold.

Datasets	Metric	GMC TKDE20	CGL TMM22	CoMSC TNNLS22	EMVGC MM23	UPGMC TNNLS24	UPCoMSC TNNLS24	TSSR TMM24	EMKIC TIP24	IMGCGGR Ours
ACM	ACC	35.17 ± 1.12	83.42 ± 2.91	78.81 ± 2.44	46.35 ± 8.94	72.03 ± 0.31	50.82 ± 20.93	40.05 ± 6.63	51.78 ± 15.02	91.37 ± 0.05
	NMI	0.33 ± 0.02	52.32 ± 6.24	44.32 ± 3.42	7.78 ± 8.79	36.62 ± 0.11	16.01 ± 22.11	4.44 ± 6.97	21.97 ± 21.03	70.12 ± 0.04
	ARI	0.04 ± 0.01	57.65 ± 6.49	48.40 ± 4.37	7.84 ± 8.99	38.17 ± 0.08	17.72 ± 24.79	3.97 ± 5.96	19.33 ± 19.40	76.14 ± 0.09
	F1	17.53 ± 0.68	83.42 ± 2.93	78.79 ± 2.51	45.51 ± 9.05	71.96 ± 0.23	50.70 ± 21.02	32.79 ± 13.70	48.69 ± 17.75	91.38 ± 0.05
AMP	ACC	27.22 ± 2.10		57.60 ± 7.30	27.31 ± 11.86	61.09 ± 1.52	50.62 ± 16.84	16.66 ± 4.12	24.74 ± 4.56	77.04 ± 0.64
	NMI	4.06 ± 0.03	eigs	46.20 ± 7.19	11.06 ± 14.62	49.42 ± 0.52	37.01 ± 18.29	0.27 ± 0.28	5.01 ± 6.80	66.05 ± 1.24
	ARI	−0.40 ± 0.01	error	33.59 ± 8.69	7.65 ± 10.60	39.12 ± 1.02	28.92 ± 14.84	-0.06 ± 0.13	2.08 ± 2.75	57.12 ± 0.35
	F1	7.81 ± 0.34		55.51 ± 7.08	23.86 ± 12.88	60.49 ± 1.63	47.95 ± 16.93	12.06 ± 2.07	15.61 ± 8.87	69.05 ± 3.55
Citeseer	ACC		29.71 ± 5.02	42.39 ± 7.25	38.67 ± 10.84	47.06 ± 0.47	43.67 ± 3.35	21.20 ± 0.62	26.61 ± 7.52	66.71 ± 1.81
	NMI	evaluation	7.52 ± 4.33	20.39 ± 6.40	15.83 ± 8.87	21.68 ± 0.14	20.10 ± 3.74	0.99 ± 0.54	5.39 ± 6.41	39.45 ± 1.34
	ARI	error	5.69 ± 3.37	17.27 ± 5.91	13.36 ± 8.99	20.53 ± 0.09	17.48 ± 3.51	0.11 ± 0.10	4.29 ± 5.31	40.40 ± 2.07
	F1		24.44 ± 5.22	38.47 ± 10.34	34.29 ± 15.32	44.33 ± 0.56	41.07 ± 2.78	10.86 ± 3.26	20.00 ± 12.22	59.22 ± 0.19
Cora	ACC	36.52 ± 2.03	41.73 ± 3.33	42.81 ± 7.37	29.49 ± 4.28	41.11 ± 0.23	38.27 ± 6.86	24.55 ± 5.30	30.24 ± 0.01	68.54 ± 0.17
	NMI	13.49 ± 0.89	24.31 ± 2.19	19.89 ± 8.41	11.76 ± 4.37	18.89 ± 0.27	17.33 ± 5.56	0.73 ± 0.32	0.40 ± 0.01	50.46 ± 0.60
	ARI	2.88 ± 0.01	17.84 ± 3.07	15.34 ± 6.88	6.16 ± 3.07	14.70 ± 0.55	13.66 ± 5.20	−0.08 ± 0.38	−0.02 ± 0.01	44.84 ± 0.25
	F1	20.67 ± 1.04	37.80 ± 4.64	36.72 ± 11.52	26.68 ± 4.56	37.64 ± 1.07	35.09 ± 6.07	11.99 ± 3.76	6.84 ± 0.01	60.49 ± 0.22
DBLP	ACC		31.73 ± 2.42	48.49 ± 12.40	37.86 ± 4.12	58.49 ± 0.38	40.79 ± 10.90	26.80 ± 1.17	28.06 ± 0.01	74.00 ± 1.18
	NMI	eigs	2.31 ± 1.42	17.51 ± 11.16	8.43 ± 4.09	25.05 ± 0.33	11.83 ± 10.20	0.19 ± 0.26	0.18 ± 0.01	40.43 ± 1.53
	ARI	error	1.94 ± 1.29	15.81 ± 10.53	5.95 ± 2.90	22.12 ± 0.28	10.08 ± 8.87	0.09 ± 0.21	0.23 ± 0.16	43.44 ± 1.96
	F1		30.50 ± 2.54	43.23 ± 20.24	36.76 ± 4.73	58.18 ± 0.36	39.74 ± 11.86	26.46 ± 1.28	23.86 ± 2.97	73.82 ± 1.19
Pubmed	ACC	39.99 ± 0.55		58.16 ± 4.30	48.78 ± 3.95	59.55 ± 0.04	49.81 ± 9.79	38.96 ± 6.12	40.26 ± 4.05	63.97 ± 2.41
	NMI	3.37 ± 0.69	eigs	21.24 ± 4.86	11.04 ± 3.83	22.61 ± 0.04	10.35 ± 8.71	2.11 ± 2.85	1.49 ± 2.75	24.09 ± 2.37
	ARI	−1.67 ± 0.34	error	19.62 ± 3.83	8.84 ± 3.27	21.05 ± 0.05	9.96 ± 8.51	2.35 ± 3.29	1.29 ± 2.53	23.94 ± 2.49
	F1	24.87 ± 0.27		57.79 ± 4.64	49.34 ± 4.75	60.19 ± 0.05	46.79 ± 11.83	37.07 ± 4.78	31.73 ± 10.27	63.78 ± 2.92

and

Table 4. The clustering performance comparison with deep multi-view graph clustering baseline methods on six benchmark datasets (mean% ± std%). The best results are highlighted in bold.

Datasets	Metric	MAGCN IJCAI21	SGCMC TMM22	GMGEC TMM23	IMGCGGR Ours
ACM	ACC	73.02 ± 16.23	69.50 ± 11.05	40.24 ± 4.90	91.37 ± 0.05
	NMI	49.54 ± 21.12	37.11 ± 9.76	4.10 ± 4.86	70.12 ± 0.04
	ARI	49.24 ± 26.32	34.20 ± 14.33	2.53 ± 2.84	76.14 ± 0.09
	F1	68.83 ± 19.66	68.77 ± 13.07	33.56 ± 10.39	91.38 ± 0.05
AMP	ACC	31.50 ± 12.08	71.20 ± 4.45	29.36 ± 3.00	77.04 ± 0.64
	NMI	12.09 ± 21.93	61.43 ± 1.95	18.35 ± 6.46	66.05 ± 1.24
	ARI	7.08 ± 14.55	54.32 ± 3.35	10.17 ± 3.89	57.12 ± 0.35
	F1	14.67 ± 17.71	61.51 ± 5.99	21.58 ± 5.79	69.05 ± 3.55
Citeseer	ACC	58.01 ± 5.92	43.99 ± 5.52	31.80 ± 3.20	66.71 ± 1.81
	NMI	36.59 ± 3.32	28.83 ± 2.67	10.26 ± 2.16	39.45 ± 1.34
	ARI	35.54 ± 4.88	14.20 ± 4.70	6.35 ± 1.53	40.40 ± 2.07
	F1	48.49 ± 7.28	39.21 ± 5.91	27.25 ± 3.10	59.22 ± 0.19
Cora	ACC	64.20 ± 3.28	66.67 ± 3.02	43.67 ± 5.41	68.54 ± 0.17
	NMI	48.65 ± 2.99	49.50 ± 1.63	28.30 ± 4.94	50.46 ± 0.60
	ARI	43.27 ± 3.11	41.63 ± 4.76	20.21 ± 5.10	44.84 ± 0.25
	F1	51.93 ± 5.31	54.80 ± 4.99	33.35 ± 6.91	60.49 ± 0.22
DBLP	ACC	40.76 ± 2.66	61.23 ± 5.60	39.83 ± 4.58	74.00 ± 1.18
	NMI	10.06 ± 3.62	32.87 ± 3.11	8.28 ± 3.32	40.43 ± 1.53
	ARI	4.83 ± 0.96	26.02 ± 5.23	6.89 ± 2.46	43.44 ± 1.96
	F1	29.53 ± 5.39	57.92 ± 7.73	32.49 ± 6.83	73.82 ± 1.19
Pubmed	ACC	56.59 ± 4.26	52.78 ± 9.10	38.24 ± 1.43	63.97 ± 2.41
	NMI	20.90 ± 4.97	17.25 ± 8.88	2.10 ± 1.13	24.09 ± 2.37
	ARI	21.19 ± 4.12	13.43 ± 11.08	1.06 ± 0.98	23.94 ± 2.49
	F1	43.30 ± 8.82	46.64 ± 14.02	33.84 ± 3.74	63.78 ± 2.92

present the clustering performance comparison results on all benchmark datasets. We have bolded the best results for each dataset. The numbers in the tables are the average evaluation metrics and the corresponding standard deviation. It can be observed that our proposed IMGCGGR achieves the best performance on all datasets. Specifically, taking the results on AMP for example, IMGCGGR significantly outperforms UPGMC and SGCMC by 26.11% and 8.20% ACC, respectively. These results demonstrate that IMGCGGR can effectively learn clustering-friendly graph representations compared to traditional graph clustering and deep graph clustering methods, thereby achieving superior clustering performance.

In addition, we observe that some traditional multi-view clustering methods encounter errors when running on multi-view graph data. The “evaluation error” indicates that although the GMC method obtained clustering results on the Citeseer dataset, these results assigned the majority of samples to a single class, and the total number of clusters exceeded the true number of categories. The “eigs error” refers to the fact that both the GMC and CGL methods encountered eigenvalue decomposition errors. The primary reason for these errors is that multi-view graph data are mostly sparse. Traditional multi-view clustering methods are more prone to errors when performing iterative optimization on sparse data. Moreover, compared to representative deep multi-view graph clustering methods, IMGCGGR also achieves the best performance. The main reason is that they are based on a standard graph convolution network, which fail to capture global graph structural information. Their representation fusion strategies lack flexibility and reliable guidance.

4.3. Ablation Studies

To thoroughly validate the effectiveness of each module, we compare IMGCGGR with its four variants on six datasets.

Table 5. The ablation studies results (mean% ± std%). The best results are highlighted in bold.

Dataset	Metrics	noGSA	noSelf	noGuide	noCrossView	IMGCGGR
ACM	ACC	89.52 ± 0.41	41.12 ± 8.80	81.73 ± 0.75	88.60 ± 0.33	91.37 ± 0.05
	NMI	68.30 ± 0.73	6.29 ± 8.84	52.47 ± 0.52	62.78 ± 0.90	70.12 ± 0.04
	ARI	71.96 ± 0.96	5.03 ± 9.02	55.46 ± 1.53	69.04 ± 0.83	76.14 ± 0.09
	F1	89.35 ± 0.42	26.62 ± 11.91	81.74 ± 0.70	88.65 ± 0.33	91.38 ± 0.05
AMP	ACC	66.58 ± 5.64	26.13 ± 0.03	68.23 ± 9.14	62.36 ± 1.80	77.04 ± 0.64
	NMI	54.23 ± 4.39	0.67 ± 0.01	58.98 ± 8.79	48.70 ± 1.43	66.05 ± 1.24
	ARI	45.84 ± 5.26	−0.11 ± 0.02	49.73 ± 10.82	39.81 ± 1.04	57.12 ± 0.35
	F1	56.09 ± 9.14	6.13 ± 0.01	57.08 ± 10.69	49.28 ± 5.25	69.05 ± 3.55
Citeseer	ACC	62.45 ± 3.76	41.44 ± 5.35	53.60 ± 2.23	60.77 ± 5.61	66.71 ± 1.81
	NMI	37.94 ± 2.51	20.95 ± 3.05	25.36 ± 2.16	36.13 ± 4.58	39.45 ± 1.34
	ARI	36.70 ± 3.58	16.97 ± 3.15	24.88 ± 2.28	34.46 ± 6.41	40.40 ± 2.07
	F1	58.23 ± 3.29	28.25 ± 7.61	50.13 ± 1.48	56.67 ± 6.25	59.22 ± 0.19
Cora	ACC	64.02 ± 2.38	40.37 ± 3.09	61.99 ± 3.08	58.81 ± 8.83	68.54 ± 0.17
	NMI	46.05 ± 3.37	20.81 ± 2.72	44.12 ± 3.27	39.29 ± 8.70	50.46 ± 0.60
	ARI	38.56 ± 3.52	15.27 ± 2.97	40.77 ± 4.18	34.58 ± 7.33	44.84 ± 0.25
	F1	53.26 ± 5.92	25.69 ± 4.75	53.39 ± 4.36	48.63 ± 10.82	60.49 ± 0.22
DBLP	ACC	70.82 ± 2.22	40.39 ± 2.16	52.72 ± 2.10	69.93 ± 2.94	74.00 ± 1.18
	NMI	36.77 ± 2.92	10.14 ± 2.60	20.21 ± 1.56	36.16 ± 3.69	40.43 ± 1.53
	ARI	38.27 ± 3.80	9.11 ± 2.83	18.61 ± 1.55	37.57 ± 3.72	43.44 ± 1.96
	F1	70.81 ± 2.17	32.34 ± 2.93	50.02 ± 2.00	69.51 ± 3.32	73.82 ± 1.19
Pubmed	ACC	57.58 ± 4.64	39.95 ± 0.01	58.96 ± 3.22	60.09 ± 3.90	63.97 ± 2.41
	NMI	21.64 ± 2.04	0.02 ± 0.01	18.12 ± 2.87	21.64 ± 2.03	24.09 ± 2.37
	ARI	21.35 ± 1.37	0.01 ± 0.01	15.80 ± 2.79	21.35 ± 1.37	23.94 ± 2.49
	F1	56.83 ± 8.20	19.04 ± 0.01	59.04 ± 3.40	56.83 ± 8.20	63.78 ± 2.92

reports the ablation experiment results. Concretely, “noGSA” means a variant with the global self-attention module being removed. As reflected in Equation (4), this implies the absence of the term $\beta {\mathit{o}}^{\left(\mathit{v}\right)}$. “noSelf” denotes the network trained without ${\mathcal{L}}_{self}$ loss. As reflected in Equation (13), this implies the absence of the term ${\lambda }_1{\mathcal{L}}_{self}$. “noGuide” indicates the network trained without ${\mathcal{L}}_{guide}$ loss. As reflected in Equation (13), this implies the absence of the term ${\lambda }_2{\mathcal{L}}_{guide}$. “noCrossView” represents that we replace the adaptive cross-view fusion module with a simple view accumulation.

Table 5 shows that all modules in IMGCGGR contribute to performance. The ${\mathcal{L}}_{self}$ loss is most critical, as removing it (“noSelf”) leads to very poor clustering due to missing soft assignment guidance from the auxiliary distribution. This is fatal for the training of the entire model. The global self-attention module significantly improves results by capturing global information via the self-attention mechanism. The ${\mathcal{L}}_{guide}$ loss is essential for aligning attribute and structural representations through the auxiliary distribution. The absence of this loss is bound to significantly impact the training of the network and the learning of the fused representation. Meanwhile, observing the comparison between (“noCrossView”) and IMGCGGR, it is evident that the weights of different views also influence the generation of the final consensus representation. If the importance of views is not considered and they are treated equally by direct summation, it will significantly degrade the model’s final performance.

4.4. Hyper-Parameter Sensitivity Analysis

In our model, hyper-parameters ${\lambda }_1,{\lambda }_2$ are the trade-off parameters for ${\mathcal{L}}_{self}$ and ${\mathcal{L}}_{guide}$ loss. To investigate the impact of hyper-parameters on model performance, we perform comprehensive sensitivity experiments. Specifically, we evaluate IMGCGGR by varying ${\lambda }_1$ and ${\lambda }_2$ from ${10}^{-3}$ to ${10}^3$. Figure 3 shows the hyper-parameter sensitivity results on ACM, and we can see that the model performance is optimized when ${\lambda }_1$ and ${\lambda }_2$ are set to 1 and 10. When ${\lambda }_2<1$, the model performance degrades as ${\lambda }_2$ decreases. This demonstrates the crucial role of the guide clustering loss in representation learning. The ${\mathcal{L}}_{guide}$ term not only guides the training of view-specific representations ${\mathbf{Z}}^{\left(v\right)}$ but also establishes a connection between the attribute distribution and structural distribution for joint optimization. For the remaining datasets, we also conducted similar experiments on the sensitivity of hyper-parameters. The detailed setting results of hyper-parameters ${\lambda }_1$ and ${\lambda }_2$ are shown in

Table 6. The hyper-parameters setting for all datasets.

Dataset	${\mathit{\lambda }}_1$	${\mathit{\lambda }}_2$
ACM	1	1
AMP	1	1
Citeseer	1	1
Cora	1	10
DBLP	1	1
Pubmed	1	1

. When applying our IMGCGGR method to other datasets, typically, we recommend ${\lambda }_1={\lambda }_2=1$ for most datasets.

4.5. Running Time

We present the runtime comparison on the largest Pubmed dataset in Figure 4 with the y-axis on a ${log}_{10}$ scale. Our method shows competitive efficiency on large datasets, outperforming most benchmarks. Although CoMSC and EMVGC exhibit faster runtimes, our approach consistently delivers superior performance with acceptable time. Additionally, the running time of CGL is missing because there was an error in calculating the eigs on the Pubmed data. The CGL algorithm failed to run properly and thus could not obtain the running time result.

4.6. Convergence Analysis

The convergence curve using ACM as an example in Figure 5 demonstrates a monotonically decreasing trend of the objective function value followed by stabilization, which confirms the convergence of the model.

4.7. Visualization Analysis

In order to show the superiority of our method, we utilize t-SNE to visualize the raw multi-view feature matrix and the fused view-specific representation on the ACM and AMP datasets. The visualization results are shown in Figure 6. By comparing the visualization results of the raw features with those of the fused representation, it is evident that the representation obtained through the view-specific fusion network achieves a more distinct separation of different clusters.

Furthermore, to demonstrate our model’s superior representation learning, we visualize the consensus representations learned by different competitive methods on ACM. We select four baseline methods (CGL, CoMSC, EMVGC, UPGMC) for a visual comparison in Figure 7. From the visualization results, it can be seen that compared with other methods, our method achieves clearer representation separation, denser clusters, and better clustering performance.

5. Conclusions

In this paper, we present a novel multi-view clustering algorithm called improved multi-view graph clustering with global self-attention (IMGCGGR) to address the multi-view graph-structured data clustering task. To fully capture and flexibly integrate the node attribute and structural information, we propose a view-specific fusion network. The view-specific fusion network not only utilizes the global self-attention mechanism to enhance the global properties of structural information but also employs a self-supervised strategy to guide the training of the clustering distribution assignment. Furthermore, taking into account the importance of different views, we exploit the cross-view fusion module to adaptively weight the fused view-specific representations, thereby generating a more effective final consensus representation. Although our model is surprisingly simple, extensive elaborate experimental results show its effectiveness. In particular, it surpasses or is comparable to the best methods recently proposed in standard benchmarks, demonstrating its effectiveness and competitive performance. The limitations and potential deficiencies of our algorithm lie in its inability to leverage performance advantages for multi-view data lacking structural information while also being inapplicable to datasets with incomplete attribute view. In future works, we will try to extend the proposed method to a more complex scenario, such as incomplete multi-view clustering and incomplete graph clustering tasks.