Cluster Complementarity and Consistency Mining for Multi-View Representation Learning

Abstract

With the advent of the big data era, multi-view clustering (MVC) methods have attracted considerable acclaim due to their capability in handling the multifaceted nature of data, which achieves impressive results across various fields. However, two significant challenges persist in MVC methods: (1) They resort to learning view-invariant information of samples to bridge the heterogeneity gap between views, which may result in the loss of view-specific information that contributes to pattern mining. (2) They utilize fusion strategies that are susceptible to the discriminability of views, i.e., the concatenation and the weighing fusion of cross-view representations, to aggregate complementary and consistent information, which is difficult to guarantee semantic robustness of fusion representations. To this end, a simple yet effective cluster complementarity and consistency learning framework (CommonMVC) is proposed for mining patterns of multiview data. Specifically, a cluster complementarity learning is devised to endow fusion representations with discriminate information via nonlinearly aggregating view-specific information. Meanwhile, a cluster consistency learning is introduced via modeling instance-level and cluster-level partition invariance to coordinate the clustering partition of various views, which ensures the robustness of multi-view data pattern mining. Seamless collaboration between two components effectively enhances multi-view clustering performance. Finally, comprehensive experiments on four real-world datasets demonstrate CommonMVC establishes a new state-of-the-art baseline for the MVC task.

1. Introduction

With the proliferation of multimedia technologies, a vast amount of data generated by various sensors and extractors has emerged in recent years [1,2]. For instance, in the traffic analysis task, data usually are collected from ultrasonic radars, optical cameras, and infrared radars. In healthcare, data can be obtained from various diagnostic tools such as MRI scans, CT scans, and X-rays. These data are commonly called multi-view data, and each view exhibits consistency that expresses common properties of samples and complementarity that expresses specific properties of samples. Effective aggregation of consistent and complementary information can endow data with discriminative and expressive abilities, which greatly facilitate the mining of data patterns in the downstream tasks [3,4].

As one of the most important multi-view information fusion paradigms, multi-view clustering (MVC) has received increasing attention, which is divided into two categories based on differences among fusion strategies of consistent and complementary information, i.e., the multi-view concatenation fusion method [5,6,7,8] and the multi-view weighing fusion method [9,10,11,12]. The former directly concatenates features from different views into a single fusion representation to fuse consistent and complementary information for clustering. The latter assigns different weights to features of each view based on their importance and reliability in fusing consistent and complementary information, ensuring that the most informative views contribute more to the final clustering result.

Although existing MVC methods have made encouraging progress in mining patterns of multi-view data, there are still two issues. Firstly, most MVC methods [13,14,15,16] impose alignment constraints, such as contrastive alignment and adversarial alignment, on view-specific representations to bridge the heterogeneity gap between views for aggregating consistent and complementary information. For example, MGCC proposes a multi-view granular ball contrastive clustering method via maximizing similarities between data within the same granular ball and minimizing similarities between data in different granular balls, which enforces the local structure invariance for clustering [17]. However, the essence of these alignment constraints is to explore inter-view invariant information, i.e., consistency, which may suppress complementarity information mining of multi-view data and cause the loss of useful information. Secondly, most MVC methods [12,18,19,20] utilize fusion strategies, i.e., the direct concatenation or the weighed combination of view-specific representations, to obtain view-common representations of multi-view data. For example, DMAC-SI concatenates view-specific representations with semantic invariance in the mirror encoder-decoder framework and then imposes a data-driven partitioning strategy on fusion representations for obtaining clustering results [21]. Such a manner, which is susceptible to representation discriminability in each view, may not guarantee semantic robustness of view-common representations.

In the paper, a simple and effective cluster complementarity and consistency leaning (CommonMVC) is proposed to solve the above problems. Specifically, CommonMVC defines a view-specific representation learning and cluster partitioning component via deploying encoding-decoding networks with clustering layers for each view to obtain vanilla latent representations and soft assignments of data. Then, CommonMVC designs a cluster complementarity learning via the nonlinear fusion functions to learn view-common representations, which can fully complement information between views and avoid the nuisance of certain views with fuzzy clustering structures. These nonlinear fusion functions are derived from maximizing mutual information between views, which are implemented using the multi-layer perceptron networks. Meanwhile, CommonMVC designs a cluster consistency learning via modeling instance-level and cluster-level partition invariance. The former ensures that the same sample from different views belongs to the same cluster, while the latter ensures that the same cluster in different views has a similar sample distribution. This dual-level invariance enhances the consistency and robustness of multi-view data pattern mining. In CommonMVC, the seamless collaboration of the three components in CommonMVC yields improved multi-view clustering performance with more accurate, comprehensive, and consistent clustering results in multi-view data. Ultimately, experiment results on four real-world datasets demonstrate that CommonMVC achieves state-of-the-art performance in MVC.

The main contributions of CommonMVC are threefold:

• A complementarity-leaning strategy is proposed to nonlinearly aggregate view-specific information, which ensures robustness of fusion representations.
• A consistency leaning strategy is devised via modeling instance-level and cluster-level partition invariance, to alleviate nuisances of some views with fuzzy cluster structures, which effectively enhances intrinsic pattern mining.
• Extensive evaluations conducted on four benchmarks verify that CommonMVC sets a new standard for MVC tasks compared with baseline methods.

The paper proceeds as follows: Section 2 constructs related work of CommonMVC. Section 3 constructs the methodological framework, detailing our novel architecture and adaptive mechanisms. Section 4 conducts rigorous multidimensional evaluation across benchmarks, comparative analysis, and ablation studies. Section 5 synthesizes core contributions, discusses practical implications, and delineates emergent research trajectories.

2. Related Work

Multi-view clustering refers to the methodology in data analysis and machine learning that utilizes information from multiple sources or feature sets to enhance clustering accuracy and robustness. Based on the paradigm of information fusion, MVC methods are divided into multi-view concatenation fusion methods and multi-view weighing fusion methods.

2.1. Multi-View Concatenation Fusion Method

Multi-view concatenation fusion methods combine features from different views into a single comprehensive representation by directly concatenating them, enabling unified data processing and clustering. For instance, 2CMV concatenates consistent and complementary information extracted via the matrix factorization to learn fusion representations and then performs vanilla k-means on fusion representations to obtain clustering results of multi-view data [22]. MFK concatenates consistent and complementary information within the mutual guidance between view-common cluster partitioning and view-specific coefficient matrices learning, and then employs k-means on fusion representations to obtain clustering results of multi-view data [23]. MvAGC concatenates smooth view-specific representations learned via self-expression layers with the graph filter and then utilizes spectral clustering on the consensus matrix of fusion representations for capturing clustering structures [24]. CSMSC concatenates consistent and complementary information between views disentangled via view-common and view-specific self-expression matrices for performing spectral clustering to explore data structures [25]. DMCAG defines an anchor multi-view subspace clustering to learn the consensus matrix between views, and then utilizes spectral self-supervised learning to align view-specific clustering assignment for mining patterns of data [26]. DMIM designs an over-clustering encoding-decoding network via maximizing the dependence between representations in different views and minimizing the dependence between representations and data in the same views and then performs k-means on weighted fusion representations to explore clustering patterns [11]. SDMVC concatenates view-specific representations learned via a self-supervised clustering consistency strategy to fuse complementary and consistent information, along with an end-to-end pattern mining [12].

2.2. Multi-View Weighing Fusion Method

Multi-view weighing fusion methods assign different weights to each view based on their importance and reliability, combining them to capture both consistent and complementary information for more accurate and robust clustering. For instance, CoMVC conducts a contrastive encoding-decoding network via weighted aligning view-specific complementary information to learn view-common representations and then utilizes the Cauchy-Schwarz divergence to explore structure divergences between data [27]. MVaDE conducts a variational generation network from hidden categories to real data for weighted aggregation of consistent and complementary information, where the mixture of Gaussian distributions is used to infer cluster partitions of multi-view data [7]. MSEC obtains an inter-view consensus matrix for clustering via weighted fusion of view-specific consensus matrices that are learned via view-specific metric functions, where inter-view and intra-view structure invariant constraints are used to enhance the aggregation of consistent and complementary information [8]. CMSC fuses information of views based on view importance to learn the consensus matrix in a shared subspace where the correlations between views are maximized via the canonical correlation analysis and then performs spectral clustering to explore data structures [28]. COMIC aligns view-specific graphs to aggregate consistent and complementary information and then conducts weighted partitions in a view-specific graph to mine patterns of multi-view data [10]. CMIB designs an information bottleneck encoding-decoding network to learn robust view-common representations via weighting complementary, consistent, and superfluous information among views and then performs k-means on view-common representations to obtain final clustering results [9]. AE2-Nets designs an encoding-decoding degradation network to fuse consistent and complementary information for learning fusion representations and then performs k-means to obtain final clustering results [29]. DealMVC designs a dual contrastive multi-view clustering network via aligning inter-view and intra-view feature-level similarity graphs and class-level pseudo-label graph and then enforces intra-view pseudo-label graphs to be as similar as possible to obtain clustering results [13]. VCGA devises an information-decoupled multi-view graph clustering method via exploring view-specific graphs and the consensus graphs with the help of the local and global structure exploration, which fully explores the local manifold for mining patterns of multiview data [18]. HCLS CGL designs a confidence neighbor multi-view weighing fusion via weighting the group-wise structures between data to boost the aggregation of consistent and complementary structures across different views by leveraging the confidence of neighboring relationships [30]. STMVC proposes an uncertainty refinement strategy to trustily separate multi-view representations significantly belonging to different clusters to fuse well-separated view-specific representations into high-quality discriminative representations for clustering, where the discriminative representations were weighted on the basis of view-specific uncertainties according to the Dempster-Safer theory of evidence [14]. DMAC proposes an anchor graph alignment strategy to match multi-view representations of the view-average discriminative representations for clustering by introducing a perturbation-injected learnable anchor graph, where the anchor graph convolution clustering distributions instructed the consistency refinement of discriminative multi-view representations for efficient cluster structure mining [15].

In multi-view clustering, existing fusion strategies mainly fall into concatenation fusion and weighted fusion. Concatenation fusion directly connects features from different views, which raises the dimensionality and brings in redundant info. Weighted fusion assigns weights to views based on their importance and then combines them linearly, but it has a subjective weight-allocation process and is sensitive to view-quality differences. Nonlinear fusion, as in CommonMVC, surpasses both. Using nonlinear transformation functions, CommonMVC can uncover complex relationships between views. This makes the model more robust and better at fitting data structures, improving clustering accuracy. So, nonlinear fusion is a more effective, flexible, and robust solution than concatenation and weighted fusion.

3. The Proposed Method

Consider the task of clustering a multi-view data set of n samples with V views ${\left\{x_i\right\}}_{i=1}^n={\{x_i^1\in {\mathbb{R}}^{D^1},\dots ,x_i^V\in {\mathbb{R}}^{D^V}\}}_{i=1}^n$, where $x_i^v$ denotes the v-th view data of the i-th sample $x_i$ and $D^v$ denotes the feature dimension in the v-th view. A cluster complementarity and consistency leaning (CommonMVC) is invented to achieve MVC, which consists of view-specific representation learning and cluster partitioning, cluster complementarity learning, and cluster consistency learning, as shown in Figure 1.

3.1. View-Specific Representation Learning and Cluster Partitioning

Generally, each view, describing different aspects of the same samples, contains view-specific complementary information, which can greatly enhance the clustering performance. Thus, view-specific representation learning and cluster partitioning are designed to fully explore and capture intrinsic information hidden in each view.

Representation Learning. Inspired by effectiveness in previous works [31,32,33], deep autoencoders are utilized to learn latent representations of heterogeneous multi-view data with high dimensions. The deep autoencoder for each view works as follows: the encoder compresses the original data into a lower-dimensional latent space, capturing the essential features while discarding noise and redundancy. The decoder then attempts to reconstruct the original data from this latent representation. This process helps the model learn robust and meaningful features.

Specifically, given the v-th view data of the i-th sample $x_i^v$, the encoder $En{c}^v$ is used to transform data $x_i^v$ into the latent representation $z_i^v$, i.e., $z_i^v=En{c}^v(x_i^v;{\theta }^v)$, where ${\theta }^v$ denotes the encoder parameter in the v-th view. Meanwhile, the decoder $De{c}^v$ is used to generate reconstruction data ${\overline{x}}_i^v$ via $z_i^v$, i.e., ${\overline{x}}_i^v=De{c}^v(z_i^v;{\varphi }^v)$, where ${\varphi }^v$ denotes the decoder parameter in the v-th view. To ensure that the learned representations retain as much useful information as possible from the original data, we maximize the mutual information between the original data and the corresponding representations in each view, as follows:

$$\begin{array}{c}{L}_{base}=-\sum _{v=1}^{V}maxI(X^v;Z^v)\hfill \\ =-\sum _{v=1}^V\int d{x}^{v}d{z}^{v}p(x^v,z^v)log{\displaystyle \frac{p(x^v,z^v)}{p\left(x^v\right)p\left(z^v\right)}}\hfill \\ \le \sum _{v=1}^V\int d{x}^{v}p\left(x^v\right)\int dzp\left(z^v\right|x^v)logq\left(x^v\right|z^v)\hfill \end{array}$$

(1)

where $I(X^v;Z^v)$ represents the mutual information between the original data $X^v$ and its latent representation $Z^v$ in the v-th view. Mutual information quantifies the amount of information obtained about one random variable through the other. By maximizing it, we ensure that the latent representations $Z^v$ retain as much information as possible from the original data $X^v$, which is crucial for preserving the essential characteristics and patterns in the data.

Cluster Partitioning. To avoid suboptimal representation for clustering, the parameterized nonlinear mappings based on Student’s t-distribution are constructed on the top of encoders in each view, which generates soft cluster assignments of samples in an end-to-end manner, i.e., $f^v(Z^v;C^v):Z^v\to S^v$, where $C^v=\{c_1^v,c_2^v,\dots ,c_k^v\}$ denotes the learnable parameters and $S^v$ denotes soft cluster assignments of samples in the v-th view. The generation process of $S^v$ is as follows:

$$s_{ij}^v={\displaystyle \frac{{(1+{∥z_i^v-c_j^v∥}^2)}^{-1}}{{\sum }_{l=1}^k{(1+{∥z_i^v-c_l^v∥}^2)}^{-1}}}$$

(2)

where $s_{ij}^v$ is the j-th element of $S_i^v$ and represents the probability of assigning the i-th sample to the j-th cluster in the v-th view. Cluster partitioning utilizes Student’s t-distribution to produce smooth and robust cluster assignments, where samples closer to a cluster center have higher assignment probabilities.

3.2. Cluster Complementarity Learning

In general, some information aggradation policies, i.e., concatenated and weighed combinations, are used to generate fusion representations between views via aggregating view-specific representations. However, the concatenated combination, i.e., $Z=[Z^1,Z^2,\dots ,Z^V]$, has two significant drawbacks that may distort the final fused representation and ultimately degrade performance in downstream tasks. First, it introduces view redundancy, leading to the inclusion of repetitive or irrelevant information. Second, it is susceptible to the influence of noisy views with fuzzy data structures. The weighted combination, i.e., $Z=W^1{Z}^1+W^2{Z}^2+,\dots ,+W^V{Z}^V$, where W denotes the weight matrix, is commonly regarded as a linear mapping from view-specific representations to fusion representations. Such a linear mapping between variables cannot capture complementary information, i.e., the inseparable clusters in one variable are still inseparable in other variables. Conversely, if there is complementarity between variables, the complementary information can be delivered by a nonlinear mapping.

Rooted in the above discussions, the simple and effective nonlinear transformation functions are designed to learn fusion representations, i.e., $Z^v=f^v\left(Z\right)$, where $f^v$ denotes the nonlinear mapping function of the v-th view. Then, the mutual information between Z and $Z^v$ is maximized to optimize parameters of $g^v$, which effectively aggregates inter-view complementarity learning of clustering structures, as follows:

$$\begin{array}{cc}\hfill L_{com}& =-\sum _{v=1}^{V}maxI(Z;Z^v)=-\sum _{v=1}^V\int d{z}^{v}dzp(z^v,z)log{\displaystyle \frac{p(z^v,z)}{P\left(z^v\right)P\left(z\right)}}\hfill \\ \hfill & =-\sum _{v=1}^V\int d{z}^{v}dzp(z^v,z)log{\displaystyle \frac{p\left(z^v\right|z)}{p\left(z^v\right)}}\hfill \\ \hfill & =-\sum _{v=1}^{V}H\left(z^v\right)+\int dzp\left(z\right)\int d{z}^{v}p\left(z^v\right|z)logp\left(z^v\right|z)\hfill \\ \hfill & \le \int d{z}^{v}p\left(z^v\right)\int dzp\left(z\right|z^v)logq\left(z^v\right|z)\hfill \end{array}$$

(3)

where $q\left(z^v\right|z)$ is the inferring distribution from z to $z^v$, which can be implemented by the nonlinear mapping function $g^v$.

Cluster complementarity learning utilizes the nonlinear transformation functions to capture fusion representations with more discriminative clustering structure than that of view-specific representations and can effectively relieve interference of views with fuzzy clustering structures in fusing complementary patterns, ensuring the cluster robustness in learning multi-view fusion representations.

3.3. Cluster Consistency Learning

Due to the absence of label information in multi-view clustering, it is not possible to determine which view’s clustering partition is precise. However, multi-view clustering consistency can help mitigate these challenges because it ensures that different views provide similar clustering predictions, which reduces the likelihood of relying on any single, potentially inaccurate view. By aligning the clustering results across multiple views, the overall accuracy and robustness of the clustering outcomes are enhanced. To ensure this consistency, we propose a novel cluster consistency learning via utilizing a global clustering partition to effectively harmonize the clustering predictions across different views. Specifically, multi-view cluster consistency learning consists of instance-level partition invariance and cluster-level partition invariance. The former refers to the fact that the same sample from different views should belong to the same label, and the latter refers to the fact that the same cluster in different views should satisfy the same sample distribution.

To achieve two-partition invariance in the cluster consistency learning, a global semantic distribution is obtained as follows. A parameterized $MLP$ is constructed to generate global semantics of fusion representations, i.e., $MLP(Z;C):Z\to S$, where $C=\{c_1,c_2,\dots ,c_k\}$ stands for cluster prototypes of fusion representations and S denotes the global clustering partition:

$$\begin{array}{c}\hfill S=MLP(Z;C)=\delta \left(\epsilon \right(Z,C\left)\right)\end{array}$$

(4)

where $\epsilon (·)$ stands for the membership relationship between data and clusters:

$$\begin{array}{c}\hfill q_{ij}=\epsilon (z_i,c_j)={\displaystyle \frac{1}{1+{∥z_i-c_j∥}_2^2}}\end{array}$$

(5)

where $q_{ij}$ stands for the similarity degree of the i-th sample and the j-th cluster. $\delta (·)$ transforms the similarities to the clustering partition:

$$\begin{array}{c}\hfill s_{ij}=\delta \left(s_i\right)={\displaystyle \frac{{(q_{ij}/{\sum }_j{q}_{ij})}^2}{{\sum }_j{(q_{ij}/{\sum }_j{q}_{ij})}^2}}\end{array}$$

(6)

The Kullback–Leibler divergence between the global clustering partition of fusion representations $S_{i,·}$ and local clustering partitions of view-specific representations $S_{i,·}^v$ is minimized to perform instance-level partition invariance learning, where $S_{i,·}$ and $S_{i,·}^v$ denote the i-th row of the global clustering partition and local clustering partitions, respectively.

$$\begin{array}{c}\hfill L_{ipi}=-\sum _{v=1}^V\sum _{i=1}^n\sum _{j=1}^k{s}_{ij}log{\displaystyle \frac{s_{ij}}{s_{ij}^v}}\end{array}$$

(7)

Meanwhile, the Kullback–Leibler divergence between the global sample distribution of clusters $S_{·,i}$ and local sample distribution of clusters $S_{·,i}^v$ is minimized to perform cluster-level partition invariance learning, where $S_{·,i}$ and $S_{·,i}$ denote the i-th column of the global clustering partition and local clustering partitions, respectively.

$$\begin{array}{c}\hfill L_{cpi}=-\sum _{v=1}^V\sum _{j=1}^k\sum _{i=1}^n{s}_{ij}log{\displaystyle \frac{s_{ij}}{s_{ij}^v}}\end{array}$$

(8)

After training the model, final clustering results of samples are obtained as follows:

$$\begin{array}{c}\hfill y_i=\underset{j}{argmax}\left(s_{ij}\right)\end{array}$$

(9)

3.4. The Loss Function

CommonMVC conducts the loss function L to optimize the cluster complementarity learning and cluster consistency learning, as follows:

$$L=L_{base}+\alpha L_{com}+\beta L_{ipi}+\gamma L_{cpi}$$

(10)

where $\alpha$, $\gamma$, and $\beta$ are parameters. $L_{com}$ is the complementary loss that guides cluster complementarity learning. $L_{ipi}$ and $L_{cpi}$ are the consistent losses that guide cluster consistency learning. That is, L trains CommonMVC to fully explore complementarity and consistency hidden in multi-view data, which effectively enhances intrinsic pattern mining.

The model analysis. The analysis of why $L_{ipi}$ and $L_{cpi}$ can guarantee inter-view-consistent clustering partitions is as follows: For the v-th and the l-th views, given the optimization errors ${ϵ}_v$ and ${ϵ}_l$ of $L_{ipi}^v$ and $L_{ipi}^j$, respectively, we have:

$$\begin{array}{c}\hfill 0\le KL(S_{i,·}\parallel S_{i,·}^{\phantom{l}v})=\sum _{i=1}^n\sum _{j=1}^K{s}_{ij}log{\displaystyle \frac{s_{ij}}{s_{ij}^v}}\le {\epsilon }_v\end{array}$$

(11)

$$\begin{array}{c}\hfill 0\le KL(S_{i,·}\parallel S_{i,·}^l)=\sum _{i=1}^n\sum _{j=1}^K{s}_{ij}log{\displaystyle \frac{s_{ij}}{s_{ij}^l}}\le {\epsilon }_l\end{array}$$

(12)

via subtracting Equation (11) from Equation (12), having:

$$\begin{array}{c}\hfill -{\epsilon }_v\le \sum _{i=1}^n\sum _{j=1}^K{s}_{ij}log{\displaystyle \frac{s_{ij}^v}{s_{ij}^l}}\le {\epsilon }_l\end{array}$$

(13)

When $s_{ij}$ is fixed, and ${ϵ}_v$ and ${ϵ}_l$ decrease in the optimization precessing:

$$\begin{array}{c}\hfill \sum _{i=1}^n\sum _{j=1}^K{s}_{ij}log{\displaystyle \frac{s_{ij}^v}{s_{ij}^l}}\to 0\end{array}$$

(14)

Thus, $S_{i,·}^v$ and $S_{i,·}^l$ tend to be consistent.

Consider ${\xi }_v$ and ${\xi }_l$ as the optimization error of $L_{cpi}^v$ and $L_{cpi}^j$, respectively, having:

$$\begin{array}{c}\hfill 0\le KL(S_{·,j}\left|\right|S_{·,j}^v)=\sum _{j=1}^K\sum _{i=1}^n{s}_{ij}log{\displaystyle \frac{s_{ij}}{s_{ij}^v}}\le {\xi }_v\end{array}$$

(15)

$$\begin{array}{c}\hfill 0\le KL(S_{·,j}\left|\right|S_{·,j}^l)=\sum _{j=1}^K\sum _{i=1}^n{s}_{ij}log{\displaystyle \frac{s_{ij}}{s_{ij}^l}}\le {\xi }_l\end{array}$$

(16)

via subtracting Equation (15) from Equation (16), having:

$$\begin{array}{c}\hfill -{\xi }_v\le \sum _{j=1}^K\sum _{i=1}^n{s}_{ij}log{\displaystyle \frac{s_{ij}^v}{s_{ij}^l}}\le {\xi }_l\end{array}$$

(17)

When $s_{ij}$ is fixed, and ${\xi }_v$ and ${\xi }_l$ gradually decrease in the optimization precessing:

$$\begin{array}{c}\hfill \sum _{j=1}^K\sum _{i=1}^n{s}_{ij}log{\displaystyle \frac{s_{ij}^v}{s_{ij}^l}}\to 0\end{array}$$

(18)

Thus, $S_{·,j}^v$ and $S_{·,j}^l$ tend to be consistent. Cluster consistent learning ensures multi-view clustering consistency via utilizing S to align rows and columns of each view $S^v$.

4. Experimental Evaluation

4.1. Setup

Datasets. Four public multi-view datasets, i.e., Handwritten, ORL, LandUse-21, and Scene-15, are utilized to evaluate the performance of CommonMVC. Handwritten contains 2000 samples from 10 classes, where pixel averages in 2 × 3 windows and profile correlations are used as two views. ORL comprises 400 samples belonging to 40 classes where intensity of gray level and Gabor descriptors are used as two views. LandUse-21 includes 2100 samples categorized into 21 classes, where PHOG and LBP features are used as two views. Scene-15 consists of 4485 samples spanning 15 classes, where PHOG and GIST features are used as two views. The detailed information is shown in

Table 1. The information of datasets.

Dataset	Samples	View	Class	Content
Handwritten	2000	2	10	Handwritten Digits
ORL	400	2	40	Human Faces
LandUse-21	2100	2	21	Satellite Images
Scene-15	4485	2	15	Scene Images

.

Evaluation Metrics: To evaluate the clustering performance, three common metrics, i.e., ACC, NMI, and ARI, are used in the experiments [31]. ACC represents the rate at which samples are correctly partitioned. NMI represents similarity between the prediction distribution and the ground-truth distribution of data. ARI represents the purity of predictions. The larger values of the ACC, NMI, and ARI mean the superior clustering performance.

Comparison methods. In the experiment, fourteen comparison methods are used, including MVaDE [9], CoMVC [10], DealMVC [7], GCFAggMVC [11], SDMVC [12], DCCA [34], DCCAE [35], ${\mathrm{AE}}^2$-Nets [8], MSEC [28], MGCC [17], DMAC [19], and STMVC [14]. To ensure fair comparison, all experiments are conducted under identical hardware and software environments, with hyperparameter configurations strictly following the specifications reported in each method’s original publication.

Implementation details. CommonMVC, implemented using PyTorch 2.0.1, was employed in the experiments, where the vector features of the samples were first normalized to the $[0,1]$ range before being input into the model for training. The training process utilized the Adam optimizer with a batch size of 100, maintaining a learning rate of ${10}^{-3}$ across all CommonMVC modules. The trade-off parameters $\alpha$, $\beta$, and $\gamma$ were consistently set to $[0.1,1$], respectively, for all datasets. To ensure optimal performance of the comparison methods, a grid search strategy was applied to their trade-off parameters. Each experiment is conducted five times, with the average results reported as the final outcomes to ensure a fair comparison.

4.2. Clustering Performance Evaluation

Table 2. The average clustering results on Handwritten and ORL datasets.

Dataset	Handwritten			ORL
Metric	$\mathit{ACC}$	$\mathit{NMI}$	$\mathit{ARI}$	$\mathit{ACC}$	$\mathit{NMI}$	$\mathit{ARI}$
FeatConcate	0.610	0.607	0.553	0.571	0.753	0.477
DCCA	0.663	0.660	0.614	0.597	0.778	0.502
DCCAE	0.692	0.670	0.633	0.594	0.775	0.499
${\mathrm{AE}}^2$-Nets	0.815	0.714	0.667	0.688	0.757	0.514
MVaDE	0.888	0.808	0.776	0.695	0.736	0.504
SiMVC	0.830	0.761	0.698	0.692	0.756	0.526
SDMVC	0.899	0.821	0.801	0.610	0.756	0.504
GCFAggMVC	0.828	0.717	0.666	0.650	0.835	0.509
MSEC	0.862	0.827	0.698	0.647	0.770	0.551
DealMVC	0.813	0.718	0.642	0.137	0.329	0.044
SGDMC	0.904	0.837	0.818	0.702	0.825	0.551
MGCC	0.893	0.820	0.806	0.688	0.819	0.552
DMAC	0.888	0.817	0.798	0.686	0.804	0.527
STMVC	0.910	0.840	0.820	0.692	0.818	0.546
CommonMVC	0.912	0.848	0.823	0.711	0.848	0.580

and

Table 3. The average clustering results on LandUse and Scene datasets.

Dataset	LandUse-21			Scene-15
Metric	$\mathit{ACC}$	$\mathit{NMI}$	$\mathit{ARI}$	$\mathit{ACC}$	$\mathit{NMI}$	$\mathit{ARI}$
FeatConcate	0.123	0.161	0.036	0.208	0.304	0.116
DCCA	0.155	0.232	0.044	0.362	0.289	0.109
DCCAE	0.156	0.244	0.044	0.364	0.298	0.115
${\mathrm{AE}}^2$-Nets	0.248	0.304	0.104	0.261	0.304	0.121
MVaDE	0.225	0.225	0.094	0.378	0.299	0.118
SiMVC	0.245	0.258	0.096	0.377	0.294	0.126
SDMVC	0.238	0.229	0.120	0.386	0.213	0.126
GCFAggMVC	0.240	0.242	0.115	0.286	0.205	0.124
MSEC	0.234	0.253	0.098	0.285	0.233	0.154
DealMVC	0.180	0.192	0.065	0.278	0.226	0.140
SGDMC	0.243	0.266	0.110	0.293	0.281	0.151
MGCC	0.204	0.220	0.098	0.253	0.274	0.128
DMAC	0.238	0.255	0.116	0.300	0.301	0.142
STMVC	0.223	0.242	0.103	0.292	0.307	0.146
CommonMVC	0.259	0.280	0.120	0.310	0.312	0.152

illustrate the quantitative clustering results of CommonMVC and fourteen comparison methods on four datasets in terms of ACC, NMI, and ARI. The results show that CommonMVC contains the optimal clustering performance and sets a new baseline in the multi-view clustering task. The seasons have two aspects: (1) CommonMVC designs a cluster complementarity learning via the nonlinear fusion functions to learn view-common representations, which fully captures complementary information between views and avoids the nuisance of certain views with fuzzy clustering structures. (2) CommonMVC designs a cluster consistency learning via defining a relaxed alignment on the semantic level rather than the representation level, which alleviates the tendency of view-specific representations to be the same while capturing consistent information between views during the optimization process. In addition, there are also two observations. First, the clustering results of MVC methods are better than those obtained by FeatConcate, which proves the necessity of developing multi-perspective clustering methods. Second, the performance of one-stage MVC methods outperforms two-stage MVC methods in pattern mining of data. The reason is that one-stage MVC methods capture clustering-friendly information of multi-view data, rather than common information captured by two-stage MVC methods.

Figure 2 presents the outcomes of the Nemenyi test. A detailed perusal of the figure reveals that the average rank of the CommonMVC method surpasses those of the other methods across all four datasets. This notable ranking advantage carries substantial statistical weight. It strongly suggests that the CommonMVC method holds a significant edge over its counterparts. The consistent excellence of the CommonMVC method across various datasets makes it stand out distinctively among numerous approaches. Consequently, it offers a more precious option for relevant studies and applications.

4.3. Ablation Analysis

This section conducts loss ablation and architecture ablation on Handwritten and ORL in terms of ACC, NMI, and ARI to evaluate the contribution of each component in CommonMVC. (1) The loss ablation contains three variants. Variant_1 denotes the erasure of the mutual information $I(X^v;Z^v)$. Variant_2 denotes the erasure of $D_{KL}(S_{i,·},S_{i,·}^v)$ and $D_{KL}(S_{·,j},S_{·,j}^v)$. Variant_3 denotes the erasure of the mutual information $I(Z;Z^v)$. From ablation results in

Table 4. Loss analysis on the Handwritten and ORL dataset in terms of ACC, NMI and ARI.

Dataset	Handwritten			ORL
Metric	$\mathit{ACC}$	$\mathit{NMI}$	$\mathit{ARI}$	$\mathit{ACC}$	$\mathit{NMI}$	$\mathit{ARI}$
Variant_1	0.722	0.728	0.621	0.538	0.653	0.403
Variant_2	0.725	0.709	0.614	0.630	0.745	0.497
Variant_3	0.909	0.831	0.810	0.658	0.771	0.524
CommonMVC	0.912	0.848	0.823	0.711	0.848	0.580

, we can observe that the removal of each component of the loss function results in a degradation of performance. This observation underscores the necessity of each term in the loss function in ensuring optimal performance. Hence, these findings validate the rationality and efficacy of the loss function design. (2) The view ablation conducts view ablation on Handwritten and ORL datasets. As shown in

Table 5. View analysis on the Handwritten and ORL dataset in terms of ACC, NMI and ARI.

Dataset	Handwritten			ORL
Metric	$\mathit{ACC}$	$\mathit{NMI}$	$\mathit{ARI}$	$\mathit{ACC}$	$\mathit{NMI}$	$\mathit{ARI}$
View_1	0.822	0.755	0.723	0.633	0.737	0.498
View_2	0.722	0.620	0.636	0.578	0.684	0.421
CommonMVC	0.912	0.848	0.823	0.711	0.848	0.580

, View_1 and View_2 represent the clustering results using only the first or second view, respectively. CommonMVC, utilizing both views with the nonlinear fusion strategy, outperforms individual views in all metrics. It shows the effectiveness of integrating complementary information from multiple views. This demonstrates that the nonlinear fusion of multiple views in CommonMVC can capture more comprehensive patterns than relying on a single view.

4.4. Convergence Analysis

To validate the convergence behavior of CommonMVC, we analyzed the loss values across different training epochs on four distinct datasets, as depicted in Figure 3. The graph reveals a consistent and rapid decrease in the loss across all datasets, indicating that CommonMVC efficiently optimizes its objective functions during training. Notably, the convergence trend is evident early in the training process, with the loss stabilizing after approximately 60 epochs. The rapid convergence suggests that CommonMVC not only effectively captures the underlying data structure but also requires relatively few epochs to reach an optimal solution, making it a robust and efficient model for various datasets.

4.5. Parameter Analysis

In the hyperparameter analysis, we examine the impact of three key parameters $\alpha$, $\gamma$, and $\beta$ on the performance of CommonMVC. These parameters balance different terms in the overall loss function L of CommonMVC. We perform systematic hyperparameter search experiments across the ranges of 10, 1, 0.1, 0.01, and 0.001 for all parameters. The experiments are carried out on the Handwritten dataset, with ACC as the performance metric. Figure 4 demonstrates the optimal result is gained when $\alpha$ is in the range of 0.1 to 1, and $\gamma$ and $\beta$ are within the range of 0.01 to 0.1. Under these hyperparameter settings, the model successfully balances common view learning with individual and common feature protection. This balance enabled the model to effectively extract cross-view common features while preserving individual characteristics, leading to the best ACC on the Handwritten dataset. The findings highlight the importance of selecting appropriate hyperparameters to achieve effective feature learning and clustering performance in CommonMVC.

5. Conclusions

In this paper, we introduce CommonMVC, an innovative framework for multi-view clustering designed to mine patterns within multi-view data. CommonMVC disentangles and learns the complementary and consistent information hidden in multi-view data by defining view-specific representation learning and clustering partitioning components, along with nonlinear fusion functions. This approach uniquely designs a nonlinear fusion mechanism to enhance the complementary information between views and employs a relaxed semantic-level alignment strategy to ensure similar clustering predictions across different views, thereby improving the accuracy and robustness of the clustering outcomes. Comprehensive experiments conducted on four datasets have validated that CommonMVC achieves a new baseline for MVC tasks. CommonMVC outperforms existing baseline methods across various evaluation metrics. However, we acknowledge that our method has certain limitations. First, the computational complexity of the nonlinear fusion mechanism may increase with the number of views and the complexity of the data, which could limit its scalability for very large-scale multi-view datasets. Second, our current framework assumes that all views are available during training and testing. In scenarios where views may be missing, the performance of CommonMVC could be affected. Developing strategies to handle missing views and further optimizing the computational efficiency of the nonlinear fusion process will be important directions for future research.