Multi-View Utility-Based Clustering: A Mutually Supervised Perspective

Abstract

Information in multiple views is typically symmetrical and distinctive. Multi-view clustering generally attempts to address two key issues: (1) how to mine/leverage additional useful information between different views for current view clustering, and (2) how to combine multiple clustering results from multiple views by using compatible and complementary information hidden in multiple views. In order to achieve this, we propose a multi-view clustering method, namely the multi-view utility-based clustering method (MUC), from the novel perspective of utility-based mutual supervision between different views. The proposed method, MUC, has notable merits: (1) It moves multi-view clustering from the level of feature and/or sample side to partition side information. That is, in order to handle the first issue, MUC considers how to use utility-based partition-level side information from all the other views for current view clustering. Such partition-level side information is consistent with human thinking; it is high-caliber and instructional for clustering. (2) The utility-based partition-level side information provided by other views complements current view information in a mutually supervised way. Then, we conduct multi-view clustering via a mutual supervision mode to circumvent the second issue. As a result, using an alternating optimization strategy, the objective function of MUC can be solved in a K-means-like way. Moreover, we leverage multi-view weight learning based on maximum entropy to integrate multi-view clustering results and further improve performance. The extensive experimental results on various multi-view datasets indicate that the proposed method is better than—or at least comparable to—the existing commonly used single- and multi-view clustering methods, in terms of both clustering performance and running speed.

1. Introduction

With fast advances in multimedia, a tremendous amount of unlabeled data with multiple attribute sets emerge on the Internet every day [1,2]. Today, in most scientific data analytics, in video surveillance, social computing and environmental sciences, after data available from diverse domains are preprocessed by means of various feature extractors, they often exhibit heterogeneous properties, because their features can be naturally partitioned into groups. They are called multi-view or multi-modal datasets [3]. For multi-view data, different views are different representations of the same set of objects, describing the same subject. Figure 1 gives an illustrative example of multi-view data on the same subject. It is therefore naturally desirable that we combine multiple views properly, so as to explore more useful information for improving learning performance. This leads to the emergence of the challenging machine learning problem of multi-view learning [4]. Recently, multi-view learning has attracted widespread attention, and various multi-view learning methods, such as regression [5], classification [6], and clustering [7,8,9,10,11,12,13], have emerged.

Figure 1. Multi-view data of face images.

Among the existing multi-view learning research, multi-view clustering plays an important role. Thus far, various multi-view clustering methods have been designed for multi-view data, such as Co-FKM [11], MVKKM [12], and MV-Co-FCM [13]. Unlike traditional equivalents, multi-view clustering methods improve clustering performance by leveraging the connections between different views. There are two key issues in multi-view clustering: (1) how to mine useful additional information between different views for learning, and (2) how to combine multiple clustering results from multiple views by using the hidden, compatible, and complementary information.

For the first of these issues, since clustering is not ordered, the most instructional additional information, like the labels in supervised learning, cannot be used directly to guarantee which cluster belongs to a particular class. In response to this, and inspired by [14], we tend to utilize utility-based partition-level side information to circumvent these limitations of mining additional feature and/or sample-level side information in multi-view clustering. By utility-based partition-level side information, we mean that current view data are associated with an auxiliary partition. For two-view data, this comes from the whole data partition generated by some clustering method for the other view; for multi-view data, it comes from the averaging of or majority-voting on the whole data partitions generated by some clustering methods for all the other views. Utility-based partition-level side information is also consistent with human thinking. It is high-caliber and instructional for clustering, just as humans also tend to like to divide samples directly instead of first determining the cluster centers of the samples and then using them for division.

In relation to the second key issue, there are three types of view combination methods: concatenation, distributed and centralized. Unlike these above strategies, to address this, we propose a novel, mutual, supervised learning mechanism. In fact, since multi-view data describe the same objects from different views, naturally there is complementary partition information between multiple views. Therefore, it is reasonable for us to use the utility-based partition-level side information to learn multiple views in a mutual supervision mode. More concretely, taking two-view clustering as an example, the utility-based partition-level side information from the other view is used as the auxiliary supervised information to guide current view clustering; then, the resultant clustering results will be taken as the auxiliary partition-level information for clustering the other view. This mutual supervision procedure will be carried out alternately until certain termination criteria are met.

Furthermore, as is well known, different views have different forms of expression and provide different information, so they may have differing impacts on the final clustering. Therefore, in addition to the proposed novel use of multi-view utility-based partition-level side information, we also anticipate important evaluations of each view. Inspired by the weighted clustering algorithms presented in [15], we leverage multi-view weight learning based on maximum entropy to integrate multiple results and further improve clustering performance. In this study, considering the above two multi-view clustering issues, we propose a multi-view clustering method, namely the multi-view utility-based clustering method (MUC), presented from the novel perspective of mutual supervision between different views. The extensive experimental results on various multi-view datasets indicate that the proposed MUC is better than—or at least comparable to—the existing commonly used single view and multi-view clustering methods, in both performance and running speed. Figure 2 illustrates the framework of the proposed MUC method.

Figure 2. The framework of the proposed MUC method.

The main contributions of this study are as follows:

(1)

The proposed MUC method moves multi-view clustering from the level of feature and/or sample side information to that of the partition. By leveraging the proposed utility-based partition-level side information, the clustering process of MUC is more consistent with human thinking; it is higher-caliber and more instructional than that of sample or feature-level side information-based multi-view clustering.

(2)

While existing multi-view clustering methods share the benefits of mutual supervision learning in a co-trained and/or -regularized way, the proposed MUC method achieves a novel mutual supervision mode for first guiding the clustering of each view, in the sense of utility-based partition-level side information, for combining multi-view results. The distinctive merit of such a mutual supervision mode is that, using an alternating optimization strategy, the objective function of MUC can be solved in a K-means-like way. Our theoretical analysis indicates that MUC can guarantee an enhanced multi-view clustering performance.

(3)

In addition, based on the maximum entropy, the proposed MUC method considers the weight learning of each view, so as to further improve multi-view clustering performance.

(4)

Extensive experimental studies on various multi-view datasets justify the effectiveness of the proposed MUC method, in contrast to the existing commonly used single-and multi-view clustering methods.

The rest of this paper is organized as follows. In Section 2, we briefly review related works. The proposed multi-view utility-based clustering MUC method is described and discussed in Section 3. The extensive experimental studies are detailed in Section 4 and Section 5 concludes the paper.

2. Related Works

In clustering research, efforts have been focused on multi-view clustering in order to generate better results. In what follows, we will briefly introduce related multi-view clustering methods from the perspective of the three granulation levels of the side information used for multi-view clustering [16,17], i.e., feature-, sample- and partition-level side information. The related studies are summarized in Table 1.

Table 1. Related studies of three levels of side information.

Levels of Side Information	Related Works
Feature	Bickel [7], Kailing [18], V.R. de Sa [19], Zhou [20], Blaschko [21], Chaudhuri [22], Liu [23] and Gupta [24].
Sample	Long [25], Greene [26], Pedrycz [27], Cleuziou [11] and Jiang [13].
Partition	Han [28], Kang [29] Wang [30] and Lan [31].

Feature-level side information-based methods usually mine this information from multiple views, obtaining the resultant multi-view clustering results according to a specific combination strategy. Typical methods concatenate multiple views into one, either by directly juxtaposing the feature sets or indirectly combining the proximity matrices obtained from different views. For example, Bickel and Scheffer [7] first extended the co-EM algorithm to the general multi-view EM algorithm, before developing a variety of two-view EM and spherical K-means algorithms. Kailing et al. [18] extended DBSCAN as a multi-view version. The authors of [19] proposed two-view spectral clustering, assuming that each view is independent. Zhou and Burges [20] integrated the general single-view normalized cut of multiple views and proposed a multi-view spectral clustering algorithm. In addition, some hidden feature space-based multi-view clustering methods [21,22,23,24] also belong to this class. These methods use several feature transformation techniques [21,22,23,24] to search for the hidden feature space for multiple views; this includes canonical correlation analysis (CCA), sparse subspace clustering and non-negative matrix factorization (NMF). Then, these methods mine the additional information from the original or hidden feature for multi-view clustering.

Sample-level side information-based methods usually mine this information by, for example, restricting the consistency of cluster centers. Among these methods, some methods first cluster the samples from individual views separately, before finding a solution to express a consensus among the clustering sets. For example, with the help of a distributed framework, Long et al. [25] proposed a general model for multi-view clustering. Greene and Cunningham [26] used a late integration strategy to derive their multi-view clustering algorithm for multi-view data. Other methods utilize multiple views simultaneously and in a collaborative way, in order to ascertain multi-view dataset clusters. For example, Pedrycz [27] proposed the collaborative clustering method, CoFC, based on FCM for multi-view data, introducing a family of collaborative fuzzy matrices whose elements describe the intensity of interactions between views. Cleuziou et al. [11] developed the multi-view collaborative fuzzy clustering algorithm, CoFKM. Jiang et al. [13] introduced view weights into Co-FCM, deriving the multiple weighted view Co-FCM algorithm, WV-Co-FCM. These methods utilize the consistency of multi-view clustering centers to mine more sample-level information.

When we move from sample- or feature-level side information to a higher granulation level, partition-level side information-based methods should attempt to mine additional information directly from the whole partitions—which are generated individually in each other view—to guide current view clustering. In this regard, Han et al. [28] achieved current view clustering in a self-supervised way by identifying nearby samples in the other view as positive samples; in this way, they avoided positive samples of the same class being treated as false negative current view samples, or vice versa. Kang et al. [29] proposed a unified multi-view subspace clustering model, incorporating graph learning from each view, basic partition generation, and consensus partition fusion. Wang et al. [30] proposed a novel multi-scale deep multi-view subspace clustering (MDMVSC) method, which unifies the multi-scale learning module, self-weighting fusion module and structure preserving constraint. Lan et al. [31] proposed a novel MVSC via low-rank symmetric affinity graph to mine the consistent information and angular information of different views.

Unlike the above methods, this study attempts to first take utility-based partition-level other view side information as the auxiliary supervised information in a mutually supervised way to guide current view clustering.

3. Multi-View Utility-Based Clustering: A Mutually Supervised Perspective

In this section, we first introduce the concept of utility-based partition-level side information from other views. Then, we propose the multi-view clustering MUC method, leveraging the utility-based partition-level side information in mutual supervision mode. The corresponding solution is also derived. Theoretical discussions and details regarding computational complexity are also contained in this section.

3.1. Utility-Based Partition-Level Side Information from Other Views

According to the concept of partition-level side information outlined in Section 1, the entirety of this information generated from other views can be naturally taken as supervised information and used to guide current view clustering. In contrast to the classical feature- and/or sample-level side information, partition-level side information is more consistent with human thinking, in that it is higher-caliber and more intuitive and instructional for clustering. In order to leverage the partition-level side information from other views to help current view clustering in a mutually supervised way, we define utility-based partition-level side information from other views, based on the concept of categorical utility function [32,33], as follows.

Definition 1.

Given the total number K of clusters, the partition of current view v denoted by the corresponding indicator matrix $H^{\left(v\right)}(\in {\left\{0,1\right\}}^{n\times K})$, for current view dataset $X^{\left(v\right)}$$(\in {ℝ}^{d_v\times n})$ containing n objects, we can take the partition $H^{\left(v^{\prime }\right)}$$(\in {\left\{0,1\right\}}^{n\times K})$ on all the other views as the partition-level side information for current view $v$. Note that $H^{\left(v^{\prime }\right)}$ is generated by averaging or majority-voting on the whole data partitions from all the other views. Then we term the categorical utility functions $U_c\left(H^{\left(v\right)},H^{\left(v^{\prime }\right)}\right)$ as the utility-based partition-level side information for current view $v$.

To clearly understand $U_c\left(H^{\left(v\right)},H^{\left(v^{\prime }\right)}\right)$, we introduce the contingency matrix in Table 2 in which K clusters, denoted as $C_k^{\left(v\right)}$ and $C_j^{\left(v^{\prime }\right)}$ (j = 1, 2, ⋯⋯, K) are assumed in $H^{\left(v\right)}$ and $H^{\left(v^{\prime }\right)}$, respectively. Let $n_{kj}^{\left(v\right)\left(v^{\prime }\right)}$ denoted the number of objects belonging to both the cluster centroids matrix $C_j^{\left(v^{\prime }\right)}$ in $H^{\left(v^{\prime }\right)}$ and the cluster centroids matrix $C_k^{\left(v\right)}$ in $H^{\left(v\right)}$, $n_{k+}^{\left(v\right)}={\displaystyle {\sum }_{j=1}^K{n}_{kj}^{\left(v\right)\left(v^{\prime }\right)}}$ and $n_{+k}^{\left(v^{\prime }\right)}={\displaystyle {\sum }_{j=1}^K{n}_{jk}^{\left(v\right)\left(v^{\prime }\right)}}$, $1\le k\le K$. Let $p_{kj}^{\left(v^{\prime }\right)}=n_{kj}^{\left(v\right)\left(v^{\prime }\right)}/n$, $p_{k+}^{\left(v^{\prime }\right)}=n_{k+}^{\left(v^{\prime }\right)}/n$ and $p_{+j}^{\left(v^{\prime }\right)}=n_{+j}^{\left(v^{\prime }\right)}/n$, we have a normalized contingency matrix and accordingly can define a variety of utility functions. In this study, we prefer to adopt the following widely used category utility function:

$$U_c\left(H^{\left(v\right)},H^{\left(v^{\prime }\right)}\right)={\displaystyle \sum _{k=1}^K{p}_{k+}^{\left(v^{\prime }\right)}{\displaystyle \sum _{j=1}^K{\left(\frac{p_{kj}^{\left(v^{\prime }\right)}}{p_{k+}^{\left(v^{\prime }\right)}}\right)}^2}}-{\displaystyle \sum _{j=1}^K{\left(p_{+j}^{\left(v^{\prime }\right)}\right)}^2}$$

(1)

Table 2. Contingency matrix.

$H^{\left(v^{\prime }\right)}$
		${\mathsf{C}}_1^{\left(v^{\prime }\right)}$	${\mathsf{C}}_2^{\left(v^{\prime }\right)}$	$\cdots$	${\mathsf{C}}_K^{\left(v^{\prime }\right)}$	${\displaystyle \sum }$
$H^{\left(v\right)}$	${\mathsf{C}}_1^{\left(v\right)}$	$n_{11}^{\left(v\right)\left(v^{\prime }\right)}$	$n_{12}^{\left(v\right)\left(v^{\prime }\right)}$	$\cdots$	$n_{1K}^{\left(v\right)\left(v^{\prime }\right)}$	$n_{1+}^{\left(v\right)}$
	${\mathsf{C}}_2^{\left(v\right)}$	$n_{21}^{\left(v\right)\left(v^{\prime }\right)}$	$n_{22}^{\left(v\right)\left(v^{\prime }\right)}$	$\cdots$	$n_{2K}^{\left(v\right)\left(v^{\prime }\right)}$	$n_{2+}^{\left(v\right)}$
	$⋮$	$⋮$	$⋮$	$\cdots$	$⋮$	$⋮$
	${\mathsf{C}}_K^{\left(v\right)}$	$n_{K1}^{\left(v\right)\left(v^{\prime }\right)}$	$n_{K2}^{\left(v\right)\left(v^{\prime }\right)}$	$\cdots$	$n_{KK}^{\left(v\right)\left(v^{\prime }\right)}$	$n_{K+}^{\left(v\right)}$
	${\displaystyle \sum }$	$n_{+1}^{\left(v^{\prime }\right)}$	$n_{+2}^{\left(v^{\prime }\right)}$	$\cdots$	$n_{+K}^{\left(v^{\prime }\right)}$	$n$

Note that the last term in Equation (1) is a constant when the partition-level side information $H^{\left(v^{\prime }\right)}$ is given. Since the same objects are always described by multi-view datasets, we should make the two partitions $H^{\left(v\right)}$ and $H^{\left(v^{\prime }\right)}$ as same as possible, which actually means that we should manage to maximize the utility-based partition-level side information $U_c\left(H^{\left(v\right)},H^{\left(v^{\prime }\right)}\right)$. In order to conveniently utilize the $U_c\left(H^{\left(v\right)},H^{\left(v^{\prime }\right)}\right)$ for multi-view clustering, we prove the following theorem.

Theorem 1.

Given the utility-based partition-level side information $U_c\left(H^{\left(v\right)},H^{\left(v^{\prime }\right)}\right)$, we have

$$\underset{H^{\left(v\right)}}{\max } U_c\left(H^{\left(v\right)},H^{\left(v^{\prime }\right)}\right)\iff \underset{H^{\left(v\right)}}{\min } {‖H^{\left(v^{\prime }\right)}-H^{\left(v\right)}{G}^{\left(v^{\prime }\right)}‖}_F^2$$

(2)

where $G^{\left(v^{\prime }\right)}={\left[G_1^{\left(v^{\prime }\right)},G_2^{\left(v^{\prime }\right)},\cdots ,G_K^{\left(v^{\prime }\right)}\right]}^T$, $G_k^{\left(v^{\prime }\right)}=\left({\displaystyle \frac{p_{k1}^{\left(v^{\prime }\right)}}{p_{k+}^{\left(v^{\prime }\right)}}},{\displaystyle \frac{p_{k2}^{\left(v^{\prime }\right)}}{p_{k+}^{\left(v^{\prime }\right)}}},\cdots ,{\displaystyle \frac{p_{kK}^{\left(v^{\prime }\right)}}{p_{k+}^{\left(v^{\prime }\right)}}}\right)$ is the kth row of $G^{\left(v^{\prime }\right)}$.

Proof. 

According to the Lemma 1 in [33], $G^{\left(v^{\prime }\right)}={\left[G_1^{\left(v^{\prime }\right)},G_2^{\left(v^{\prime }\right)},\cdots ,G_K^{\left(v^{\prime }\right)}\right]}^T$ is the K cluster centers of the binary partition $H^{\left(v^{\prime }\right)}$, where $G_k^{\left(v^{\prime }\right)}=\left({\displaystyle \frac{p_{k1}^{\left(v^{\prime }\right)}}{p_{k+}^{\left(v^{\prime }\right)}}},{\displaystyle \frac{p_{k2}^{\left(v^{\prime }\right)}}{p_{k+}^{\left(v^{\prime }\right)}}},\cdots ,{\displaystyle \frac{p_{kK}^{\left(v^{\prime }\right)}}{p_{k+}^{\left(v^{\prime }\right)}}}\right)$ is the kth row of $G^{\left(v^{\prime }\right)}$. Obviously, the right-hand-side of Equation (2) is the standard K-means on the binary partition $H^{\left(v^{\prime }\right)}$. The generalized form can be expressed as:

$$\underset{H^{\left(v\right)}}{\min }{\displaystyle \sum _{k=1}^K{\displaystyle \sum _{H_i^{\left(v^{\prime }\right)}\in C_k}f\left(H_i^{\left(v^{\prime }\right)},G_k^{\left(v^{\prime }\right)}\right)}}$$

(3)

where $H_i^{\left(v^{\prime }\right)}$ is ith row of the binary partition $H^{\left(v^{\prime }\right)}$, $G_k^{\left(v^{\prime }\right)}$ is the kth center, $f\left(·\right)$ denotes some distance function from a data point to a center. Here we take $f\left(·\right)$ as the Bregman divergence [34] which is a family of distance measures fitting the classic K-means. That is, $f\left(·\right)$ can be defined as:

$$f\left(\alpha ,\beta \right)=\Phi \left(\alpha \right)-\Phi \left(\beta \right)-{\left(\alpha -\beta \right)}^T\nabla \Phi \left(\beta \right)$$

(4)

where $\Phi \left(·\right)$ is a differentiable strictly-convex function.

Substituting Equation (4) into Equation (3), we have

$$\begin{array}{l}\underset{H^{\left(v\right)}}{\min }{\displaystyle \sum _{k=1}^K{\displaystyle \sum _{H_i^{\left(v^{\prime }\right)}\in C_k}f\left(H_i^{\left(v^{\prime }\right)},G_k^{\left(v^{\prime }\right)}\right)}} \\ =\underset{H^{\left(v\right)}}{\min }{\displaystyle \sum _{k=1}^K{\displaystyle \sum _{H_i^{\left(v^{\prime }\right)}\in C_k}\left(\begin{array}{l}\Phi \left(H_i^{\left(v^{\prime }\right)}\right)-\Phi \left(G_k^{\left(v^{\prime }\right)}\right)-\\ {\left(H_i^{\left(v^{\prime }\right)}-G_k^{v^{\prime }}\right)}^T\nabla \Phi \left(G_k^{\left(v^{\prime }\right)}\right)\end{array}\right)}}\\ =\underset{H^{\left(v\right)}}{\min }{\displaystyle \sum _{k=1}^K{\displaystyle \sum _{H_i^{\left(v^{\prime }\right)}\in C_k}\Phi \left(H_i^{\left(v^{\prime }\right)}\right)}}-{\displaystyle \sum _{k=1}^K{\displaystyle \sum _{H_i^{v^{\prime }}\in C_k}\Phi \left(G_k^{\left(v^{\prime }\right)}\right)}}\\ =\underset{H^{\left(v\right)}}{\min }{\displaystyle \sum _{H_i^{\left(v^{\prime }\right)}\in H^{\left(v^{\prime }\right)}}\Phi \left(H_i^{\left(v^{\prime }\right)}\right)}-n{\displaystyle \sum _{k=1}^K{p}_{k+}^{v^{\prime }}\Phi \left(G_k^{\left(v^{\prime }\right)}\right)}\end{array}$$

(5)

Since both ${\displaystyle {\sum }_{H_i^{\left(v^{\prime }\right)}\in H^{\left(v^{\prime }\right)}}\Phi \left(H_i^{\left(v^{\prime }\right)}\right)}$ and n are constants for the binary partition $H^{\left(v^{\prime }\right)}$, we have

$$\underset{H^{\left(v\right)}}{\min }{\displaystyle \sum _{k=1}^K{\displaystyle \sum _{H_i^{\left(v^{\prime }\right)}\in C_k}f\left(H_i^{\left(v^{\prime }\right)},G_k^{\left(v^{\prime }\right)}\right)}} \iff \underset{H^{\left(v\right)}}{\max }{\displaystyle \sum _{k=1}^K{p}_{k+}^{\left(v^{\prime }\right)}\Phi \left(G_k^{\left(v^{\prime }\right)}\right)}$$

(6)

According to the definition of $U_c\left(H^{\left(v\right)},H^{\left(v^{\prime }\right)}\right)$ in Equation (1), we can readily express it as

$$U_c\left(H^{\left(v\right)},H^{\left(v^{\prime }\right)}\right)\propto {\displaystyle \sum _{k=1}^K{p}_{k+}^{\left(v^{\prime }\right)}\Phi \left(G_k^{\left(v^{\prime }\right)}\right)}$$

(7)

which means

$$\underset{H^{\left(v\right)}}{\max } U_c\left(H^{\left(v\right)},H^{\left(v^{\prime }\right)}\right)\iff \underset{H^{\left(v\right)}}{\max } {\displaystyle \sum _{k=1}^K{p}_{k+}^{\left(v^{\prime }\right)}\Phi \left(G_k^{\left(v^{\prime }\right)}\right)}$$

(8)

By Equation (6) and Equation (8), we have

$$\underset{H^{\left(v\right)}}{\max } U_c\left(H^{\left(v\right)},H^{\left(v^{\prime }\right)}\right)\iff \underset{H^{\left(v\right)}}{\min }{\displaystyle \sum _{k=1}^K{\displaystyle \sum _{H_i^{\left(v^{\prime }\right)}\in C_k}f\left(H_i^{\left(v^{\prime }\right)},G_k^{\left(v^{\prime }\right)}\right)}}$$

(9)

Therefore, Theorem 1 holds true. □

According to Theorem 1, given $H^{\left(v^{\prime }\right)}$, the equivalent relationship between ${‖H^{\left(v^{\prime }\right)}-H^{\left(v\right)}{G}^{\left(v^{\prime }\right)}‖}_F^2$ and $U_c\left(H^{\left(v\right)},H^{\left(v^{\prime }\right)}\right)$ holds on $H^{\left(v\right)}$ for the vth view, since ${‖H^{\left(v^{\prime }\right)}-H^{\left(v\right)}{G}^{\left(v^{\prime }\right)}‖}_F^2+U_c\left(H^{\left(v\right)},H^{\left(v^{\prime }\right)}\right)=\mathrm{constant}$. In other words, one extra variable $G^{\left(v^{\prime }\right)}$ is introduced to capture the mapping relationship from $H^{\left(v^{\prime }\right)}$ to $H^{\left(v\right)}$.

3.2. MUC

3.2.1. Objective Function of MUC

Based on the above discussion about utility-based partition-level side information, we designed the following objective function for the proposed MUC method:

$$\begin{array}{l}\underset{H^{\left(v\right)},C^{\left(v\right)},w^{\left(v\right)}}{\arg \min }{\displaystyle \sum _{v=1}^V{w}^{\left(v\right)}\left({‖X^{\left(v\right)T}-H^{\left(v\right)}{C}^{\left(v\right)T}‖}_F^2-\eta U_c\left(H^{\left(v\right)},H^{\left(v^{\prime }\right)}\right)\right)}\\ +\lambda {\displaystyle \sum _{v=1}^V{w}^{\left(v\right)}\ln \left(w^{\left(v\right)}\right)}\\ s.t.H_{ik}^{\left(v\right)}\in \left\{0,1\right\},{\displaystyle \sum _{k=1}^K{H}_{ik}^{\left(v\right)}}=1,i=1,2,\cdots ,n.\\ {\displaystyle \sum _{v=1}^V{w}^{\left(v\right)}}=1.\end{array}$$

(10)

where the value of $H_{ik}^{\left(v\right)}$ denotes whether the ith sample belongs to the kth cluster or not; $w^{\left(v\right)}$ is the weight of the vth view; $\eta$ and $\lambda$ are two trade-off parameters; and V is the number of views.

The above objective function in Equation (10) consists of three terms. The first corresponds to the objective function of the standard K-means [35] with squared Euclidean distance for each weighted view. The second reflects mutual supervision by maximizing the utility-based partition-level side information $U_c\left(H^{\left(v\right)},H^{\left(v^{\prime }\right)}\right)$. The third considers the well-known Shannon entropy-based weighting scheme to weigh each view. We attempt to search for a solution, $H^{\left(v\right)}$, which not only captures the intrinsic structural information from the original objects, but also agrees as much as possible with the partition-level side information, $H^{\left(v^{\prime }\right)}$.

Remark 1.

The trade-off parameter $\lambda$ is used to control the influence of the Shannon entropy about the weight of each view. In general, we may set it to follow [13]. The trade-off parameter $\eta$ as the utility-based information regulation coefficient is used to control the utility-based effect of other views on current view. The bigger its value is, the more effect of the utility-based information from other views.

According to Theorem 1, we can leverage the utility-based partition-level side information in an efficient manner. This means that we can revisit the objective function in Equation (10) from the following new insight:

$$\begin{array}{l}\underset{H^{\left(v\right)},C^{\left(v\right)},w^{\left(v\right)}}{\arg \min }{\displaystyle \sum _{v=1}^V{w}^{\left(v\right)}\left({‖X^{\left(v\right)T}-H^{\left(v\right)}{C}^{\left(v\right)T}‖}_F^2+\eta {‖H^{\left(v^{\prime }\right)}-H^{\left(v\right)}{G}^{\left(v^{\prime }\right)}‖}_F^2\right)}\\ +\lambda {\displaystyle \sum _{v=1}^V{w}^{\left(v\right)}\ln \left(w^{\left(v\right)}\right)}\end{array}$$

(11)

In order to further justify the above objective function of MUC, we have the following analyses and explanations.

Remark 2.

MUC leverages the additional utility-based partition-level side information $U_c\left(H^{\left(v\right)},H^{\left(v^{\prime }\right)}\right)$ in a mutually supervised way. Such a leverage can be conveniently and directly realized in Equation (11) for multi-view clustering. Please note, when handling multi-view (more than two views) datasets,  $H^{\left(v^{\prime }\right)}$ in Equation (11) can be obtained by averaging or majority-voting on the data partitions generated by some clustering methods for all the other views.

Besides, the following Theorem 2 will reveal that MUC has a theoretical guarantee of enhanced multi-view clustering.

Theorem 2.

The leverage of utility-based partition-level side information in the mutual supervised way in Equation (11) can guarantee enhanced multi-view clustering performance.

Proof. 

By following Theorem 1, we can get:

$$\underset{H^{\left(v\right)}}{\max } U_c\left(H^{\left(v\right)},H^{\left(v^{\prime }\right)}\right)\iff \underset{H^{\left(v\right)}}{\min } {‖H^{\left(v^{\prime }\right)}-H^{\left(v\right)}{G}^{\left(v^{\prime }\right)}‖}_F^2$$

(12)

For the multi-view data $X^{\left(v\right)}$, the above transformations are equivalent to expanding the corresponding dimensions, that is, $\left[X^{\left(v\right)},H^{\left(v^{\prime }\right)}\right]$ corresponds the data of the vth view. Please note, $H^{\left(v\right)}$ and $H^{\left(v^{\prime }\right)}$ are actually the partitions of the multi-view datasets which describe the same objects.

After the above transformation, according to Equation (11), in the same cluster, any two samples $x_i$ and $x_j$ have the same expansion component after expanding the dimensions, so $d_{ij}$ (the distance of the two samples $x_i$ and $x_j$) will not change after expanding the dimension; while in different clusters, any two samples $x_i$ and $x_j$ have different expansion components, so $d_{ij}$ increases after expanding the dimensions. Therefore, the margins of samples in different clusters will be larger while those of samples in same clusters keeps unchangeable. Therefore, in such a way, the performance of multi-view clustering will be improved. In other words, Theorem 2 holds. □

3.2.2. Solution of MUC

In this subsection, we attempt to optimize the objective function of MUC. The proposed objective function in Equation (11) can clearly be transformed into two subproblems: multiple clustering and weight learning; these can be solved in alternative ways. That is to say, we update one by keeping the other fixed, repeating this process until convergence is achieved. The optimization process can be more concretely summarized as follows.

(1) Update $w^{\left(v\right)}$ with $H^{\left(v\right)}$, $H^{\left(v^{\prime }\right)}$, $C^{\left(v\right)}$ and $G^{\left(v^{\prime }\right)}$ fixed. Equation (11) can be expressed as follows:

$$\begin{array}{l}\underset{w^{\left(v\right)}}{\min }{\displaystyle \sum _{v=1}^V{w}^{\left(v\right)}{D}^{\left(v\right)}}+\lambda {\displaystyle \sum _{v=1}^V{w}^{\left(v\right)}\ln \left(w^{\left(v\right)}\right)}\\ s.t.{\displaystyle \sum _{v=1}^V{w}^{\left(v\right)}}=1.\end{array}$$

(13)

where

$$D^{\left(v\right)}={‖X^{\left(v\right)T}-H^{\left(v\right)}{C}^{\left(v\right)T}‖}_F^2+\eta {‖H^{\left(v^{\prime }\right)}-H^{\left(v\right)}{G}^{\left(v^{\prime }\right)}‖}_F^2$$

(14)

In order to make Equation (13) unconstrained, we express its Lagrange function as follows:

$$L\left(w^{\left(v\right)}\right)={\displaystyle \sum _{v=1}^V{w}^{\left(v\right)}{D}^{\left(v\right)}}+\lambda {\displaystyle \sum _{v=1}^V{w}^{\left(v\right)}\ln \left(w^{\left(v\right)}\right)}+\gamma \left({\displaystyle \sum _{v=1}^V{w}^{\left(v\right)}}-1\right)$$

(15)

where $\gamma$ is the Lagrange multiplier. By setting the derivative of L w.r.t $w^{\left(v\right)}$ to zero for each view, we obtain

$$w^{\left(v\right)}=\exp \left\{\left(\lambda -\left(D^{\left(v\right)}+\gamma \right)\right)/\lambda \right\}$$

(16)

In order for the KKT condition to be met [36], we substitute Equation (15) into the constraint ${\displaystyle {\sum }_{v=1}^V{w}^{\left(v\right)}}=1$, finally obtaining the following optimal weight for each view:

$$w^{\left(v\right)}={\displaystyle \frac{\exp \left\{-D^{\left(v\right)}/\lambda \right\}}{{\displaystyle {\sum }_{h=1}^v\exp \left\{-D^{\left(h\right)}/\lambda \right\}}}}$$

(17)

(2) Update $H^{\left(v\right)}$, $C^{\left(v\right)}$ and $G^{\left(v^{\prime }\right)}$ with fixed $w^{\left(v\right)}$. Equation (11) can equally be converted as follows:

$$\underset{H^{\left(v\right)},C^{\left(v\right)},G^{\left(v^{\prime }\right)}}{\arg \min }{\displaystyle \sum _{v=1}^V{w}^{\left(v\right)}\left({‖X^{\left(v\right)T}-H^{\left(v\right)}{C}^{\left(v\right)T}‖}_F^2+\eta {‖H^{\left(v^{\prime }\right)}-H^{\left(v\right)}{G}^{\left(v^{\prime }\right)}‖}_F^2\right)}$$

(18)

It is obviously not possible to update all variables directly to solve this subproblem. We can generally adopt an alternative optimization technique [37] with a closed solution in each iteration. Although such a solution can be adopted for simplicity, it becomes inefficient due to matrix multiplication and inversion. Furthermore, because MUC is designed for multiple views, the data are divided into too many fractured pieces; in practice, this makes operation difficult. In order to confirm the above novel mutual supervised learning mechanism, the subproblem of the current vth view in Equation (18) should be converted as follows:

$$\underset{H^{\left(v\right)},C^{\left(v\right)},G^{\left(v^{\prime }\right)}}{\arg \min }{‖X^{\left(v\right)T}-H^{\left(v\right)}{C}^{\left(v\right)T}‖}_F^2+\eta {‖H^{\left(v^{\prime }\right)}-H^{\left(v\right)}{G}^{\left(v^{\prime }\right)}‖}_F^2$$

(19)

Then, the subproblems in Equation (19) are alternately solved on multiple views until the objective function value in Equation (18) remains unchanged.

According to [15], the above subproblem in Equation (19) can be solved very efficiently in a mathematical way. Below the above subproblem can be equivalently transformed into a K-means-like optimization problem by concatenating the utility-based partition-level side information with the original objects, which actually implies that Equation (19) has the distinctive merit in the sense that it can be solved in K-means-like way.

We introduce the auxiliary matrices $A^{\left(v\right)}$ of the vth view, respectively, as follows

$$A^{\left(v\right)}={\left[X^{\left(v\right)T},\eta H^{\left(v^{\prime }\right)}\right]}^T$$

(20)

Because the partition-level side information is utilized to guide clustering in utility-based way, the centroids of K-means clustering becomes no longer the mean of the samples belonging to a certain cluster. Let ${\mathsf{B}}_k^{\left(v\right)}$ be the kth expanded centroid of the vth view:

$${\mathsf{B}}_k^{\left(v\right)}={\left[C_k^{\left(v\right)T},\eta G_k^{\left(v^{\prime }\right)}\right]}^T$$

(21)

Let us recall that in the standard K-means clustering, the centroids are calculated by arithmetic means, and its denominator denotes the number of objects in its corresponding cluster. Based on the above mathematical transformation, we finally transform Equation (19) as a K-means-like problem in Equation (22),

$$\underset{H^{\left(v\right)},B^{\left(v\right)}}{\arg \min }{‖A^{\left(v\right)T}-H^{\left(v\right)}{B}^{\left(v\right)T}‖}_F^2$$

(22)

which can be handy solved by the existing standard K-means solver.

So far, we alternately update $H^{\left(v\right)}$, $C^{\left(v\right)}$, $G^{\left(v^{\prime }\right)}$ and $w^{\left(v\right)}$ by optimizing the corresponding subproblems until Equation (4) becomes converged. Note that the proposed MUC utilizes the utility-based partition-level side information in a mutual supervision mode. After obtained from other views by a classical K-means algorithm, the utility-based partition-level side information is used as the auxiliary supervised information to guide clustering of current view, and then the resultant clustering results of current view will be taken as the auxiliary partition-level information for clustering of the other view. Such a mutual supervision procedure will be carried out alternately until certain termination criterion are met.

Based on the above-mentioned descriptions and analysis, the implementation of MUC is summarized in Algorithm 1.

Algorithm 1: MUC

Input: The multi-view dataset $X^{\left(v\right)}=\left[x_1^{\left(v\right)},x_2^{\left(v\right)},\cdots ,x_n^{\left(v\right)}\right]\in {ℝ}^{d_v\times n}$, where $x_i^{\left(v\right)}\in {ℝ}^{d_v}$. The random initialization of partition $H^{\left(v\right)}$. The total number K of clusters. The convergence threshold $\epsilon$. The parameters $\eta$ and $\lambda$.

Output: Final partition matrix H.

Procedure:

1: Generate the auxiliary matrices $A^{\left(v\right)}$ of each view according to Equation (22), where the partition-level side information $H^{\left(v^{\prime }\right)}$ is generated by averaging or majority-voting on the whole data partitions from all other views

2: while not converge do

3:        Randomly select K expanded centroids of each view $B^{\left(v\right)}$;

4:        Compute the distance matrices $D^{\left(v\right)}$ of each view by Equation (14)

5:        Update $w^{\left(v\right)}$ for each view by Equation (17)

6:        repeat

7:                Update the partition of the vth view by Equation (23)

$\left\{\begin{array}{l}{h}_{i,l}^{\left(v\right)}=1, \mathrm{for} l=\underset{k}{\arg \min }{‖A_i^{\left(v\right)}-B_k^{\left(v\right)}‖}_2^2,\\ h_{i,r}^{\left(v\right)}=0, \mathrm{for} r\ne l.\end{array}\right.$

8:                Update the centroids of the vth view by Equation (24)

$B^{\left(v\right)}={\left(H^{\left(v\right)T}{H}^{\left(v\right)}\right)}^{-1}{H}^{\left(v\right)T}{A}^{\left(v\right)};$

9:                Carry out step 7~step 8 alternately on each view.

10:         until the change value of the objective function in Equation (22) between two iterations is smaller than $\epsilon$

11: end while

12: Output the final partition matrix H obtained by averaging or majority-voting on the partitions $H^{\left(v\right)}$ of all views.

3.3. On Computational Complexity

In this subsection, we discuss the computational complexity of Algorithm 1.

According to the above optimization process, the whole computational complexity of Algorithm 1 can be divided into two parts to discuss. The first part deals with step 4~step 5, computational complexity of computing $D^{\left(v\right)}$ is about $O\left(Kn\left(d^{\left(v\right)}+K\right)\right)$. Thus, the total computational complexity for this part is about $O\left(Kn\left({\displaystyle {\sum }_{v=1}^V{d}^{\left(v\right)}+2K}\right)\right)$, where K is the total number of clusters, n and $d^{\left(v\right)}$ are the total numbers of instances and features in $X^{\left(v\right)}$, respectively. The second part deals with step 6~step 10, which can be readily solved by a K-means-like optimization. Hence, the second part shares almost the same computational complexity with standard K-means, i.e., $O\left(TKn\left({\displaystyle {\sum }_{v=1}^V{d}^{\left(v\right)}+2K}\right)\right)$, where $T$ is the total number of inner loops. Let $\tilde{T}$ be the total number of the outer loops in the algorithm; we immediately derive that its whole computational complexity becomes $\tilde{T}$ times the sum of the first and second parts, i.e., $O\left(\tilde{T}\left(\left(T+1\right)Kn\left({\displaystyle {\sum }_{v=1}^V{d}^{\left(v\right)}+2K}\right)\right)\right)$.

4. Experimental Studies

In this section, we present our extensive experimental results on various multi-view datasets, i.e., one synthetic, seven multi-view UCI and two real-life image datasets, to verify the performance of the proposed MUC method. We likewise present a comparative study with two commonly used single-view clustering methods, i.e., K-means [35] and ComBKM [38], and six state-of-the-art multi-view clustering methods, i.e., Co-FKM [11], MVKKM [12], MVSpec [12], WV-Co-FCM [13], TW-K-means [9], and MVASM [10]. These were assessed according to the two clustering performance indices, NMI and RI, for ten runs. The experiments were conducted using MATLAB R2016a on a computer with Intel Core i5-3317 1.70 GHz CPU and 16 GB RAM.

4.1. Datasets

4.1.1. Synthetic Dataset

In this study, we construct a three-dimensional synthetic dataset that contains three clusters with a total of 600 samples (200 samples per cluster). In order to generate different multi-view datasets, we map the synthetic one into 2D subspaces. The 3D original synthetic dataset, and its corresponding 2D equivalents, are shown in Figure 3.

Figure 3. Synthetic dataset (three different colored dots represent three clusters in each sub-figure). (a) Original 3-D dataset and (b) view 1 (X, Y), (c) view 2 (X, Z), and (d) view 3 (Y, Z) datasets.

4.1.2. UCI Datasets

Seven multi-view datasets from the frequently used UCI [39] data repositories, of which details are listed in Table 3, were adopted in the experiments.

Table 3. Multi-view datasets from UCI used in the experiments.

Datasets	Number of Samples	Number of Classes	View 1		View 2
			Description	Number of Features	Description	Number of Features
Dermatology	366	6	Histopathological view about histopathological information of a case	12	Clinical view about clinical information of a case	22
Forest Type	326	4	Image band view about image band information of the data	9	Spectrum view about spectrum values and the difference values	18
Image Segmentation	2310	7	Shape view about shape information of an image	9	RGB view about RGB information of an image	10
Iris	150	3	Sepal view about sepal length and width of an iris	2	Petal view about petal length and width of an iris	2
Multiple Features	2000	10	Fourier coefficients view about Fourier coefficients of the character shapes	76	Zernike moments view about Zernike moments of the character shapes	47
SPECTF	267	2	Stress view about single proton emission computed tomography image of heart in stress	22	Rest view about single proton emission computed tomography image of heart in rest	22
Water Treatment	527	13	Input conditions and input demands	31	Output demands	7

4.1.3. Real-Life Image Datasets

(1)

The CMU PIE image dataset [40] consists of a total of 41,368 facial images (64 × 64 in pixels) from 68 persons. The common versions of CMU PIE consist of five poses (views), i.e., C05 (left), C07 (upward), C09 (downward), C27 (frontal), and C29 (right), as organized by Deng Cai. In each view, 120 images of five different persons are randomly selected. Figure 4 illustrates images of one person from these five views. In order to make full use of the CMU PIE image dataset, we generate five combinational multi-view datasets (i.e., D1, D2, D3, D4 and D5) of the above five views, the details of which are listed in Table 4.

Figure 4. Facial images of one person from five views.

Table 4. Multi-view datasets from CMU PIE image dataset used in the experiments.

Datasets	Combination of Views
D1	C05, C07, C09, C27
D2	C05, C07, C09, C29
D3	C05, C07, C27, C29
D4	C05, C09, C27, C29
D5	C07, C09, C27, C29

(2)

The AwA image dataset [41] consists of 30,475 images of 50 animals with six pre-extracted feature representations (views) each. In this study, we randomly select eight classes (5133 images) for experiments.

4.2. Evaluation Criteria

In this study, we take the following two commonly used evaluation criteria, i.e., the Normalized mutual information (NMI) [13] and Rand index (RI) [9], to evaluate the clustering results. These are defined as follows:

Normalized mutual information (NMI)

$$NMI={\displaystyle \frac{{\displaystyle \sum _i{\displaystyle \sum _j{N}_{ij}}}\log \left(N\times N_{ij}/N_i\times N_j\right)}{\sqrt{\left({\displaystyle \sum _i{N}_i\log \left(N_i\times N\right)}\right)\times \left({\displaystyle \sum _j{N}_j\log (N_j\times N)}\right)}}}$$

(23)

Rand index (RI)

$$RI={\displaystyle \frac{{\displaystyle \sum _{ij}\left(\begin{array}{c}{N}_{ij}\\ 2\end{array}\right)-\left[{\displaystyle \sum _i\left(\begin{array}{c}{A}_i\\ 2\end{array}\right){\displaystyle \sum _i\left(\begin{array}{c}{B}_j\\ 2\end{array}\right)}}\right]}I\left(\begin{array}{c}N\\ 2\end{array}\right)}{\frac{1}{2}\left[{\displaystyle \sum _i\left(\begin{array}{c}{A}_i\\ 2\end{array}\right)+{\displaystyle \sum _i\left(\begin{array}{c}{B}_j\\ 2\end{array}\right)}}\right]-\left[{\displaystyle \sum _i\left(\begin{array}{c}{A}_i\\ 2\end{array}\right){\displaystyle \sum _i\left(\begin{array}{c}{B}_j\\ 2\end{array}\right)}}\right]I\left(\begin{array}{c}N\\ 2\end{array}\right)}}$$

(24)

where

$$\left(\begin{array}{c}N\\ K\end{array}\right)={\displaystyle \frac{N!}{K!\left(N-K\right)!}}, A_i={\displaystyle \sum _j{N}_{ij}}, B_j={\displaystyle \sum _i{N}_{ij}}$$

For a dataset with N objects, N_ij denotes the number of common objects between cluster i generated by an approach and the jth ground-truth category. N_i and N_j denote the numbers of objects in cluster i and cluster j, respectively. Note that the values of NMI and RI are from [0, 1] and $\left[-1, 1\right]$, respectively. The higher their values, the better the performance of the corresponding clustering method.

4.3. Adopted Methods and Parameters Settings

We experimentally compared MUC with two commonly used single-view clustering methods, i.e., K-means [35] and ComBKM [38], and six state-of-the-art multi-view clustering methods, i.e., Co-FKM [11], MVKKM [12], MVSpec [12], WV-Co-FCM [13], TW-K-means [9], and MVASM [10]. For K-means and ComBKM, single-view datasets were directly generated by combining features from all views. Following the guidelines presented in [9,11], and [13], and according to our extensive experiments, we used the grid search strategy to determine the appropriate parameters involved in all the adopted methods. The parameter settings for all these clustering methods are given in Table 5.

Table 5. Parameter settings of all the adopted methods.

Methods	Parameter Settings for Grid Search
K-means	The total number K of clusters.
ComBKM	The total number K of clusters.
Co-FKM	The fuzzifier $m\in \left\{1.1,1.2,\cdots ,2.8,2.9,3\right\}$; the tradeoff parameter $\eta \in \left[0,K-1/K\right]$ with a step size of 0.05, where K denotes the number of clusters.
MVKKM	The total number K of clusters., the tradeoff parameter $p\in \left\{{10}^{-7},{10}^{-6},\cdots {10}^0,\cdots ,{10}^6,{10}^7\right\}$
MVSpec	The total number K of clusters, the tradeoff parameter $p\in \left\{{10}^{-7},{10}^{-6},\cdots {10}^0,\cdots ,{10}^6,{10}^7\right\}$
WV-Co-FCM	the fuzzifier $m\in \left\{1.1,1.2,\cdots ,2.8,2.9,3\right\}$; the tradeoff parameter $\eta \in \left[0,K-1/K\right]$ with a step size of 0.05, where K denotes the number of clusters; the tradeoff parameter $\lambda \in \left\{{10}^{-7},{10}^{-6},\cdots {10}^0,\cdots ,{10}^6,{10}^7\right\}$
TW-K-means	The total number K of clusters, the tradeoff parameters $\lambda \in \left\{1,2,\cdots ,28,29,30\right\}$ and $\eta \in \left\{10,20,\cdots ,100,110,120\right\}$
MVASM	The tradeoff parameters $\gamma \in \left[0,6\right]$ with a step size of 0.1; the tradeoff parameter $q\in \left[1,6\right]$ with a step size of 0.01
MUC	The total number K of clusters, the tradeoff parameters $\eta \in \left\{0.0001,0.001,0.01,0.1,1.0,10.0,100.0\right\}$, $\lambda \in \left\{{10}^{-7},{10}^{-6},\cdots {10}^0,\cdots ,{10}^6,{10}^7\right\}$

4.4. Experimental Results

4.4.1. Synthetic Dataset

We used the proposed MUC and seven other methods on the synthetic dataset to examine effectiveness in improving multi-view clustering performance, utilizing utility-based partition-level side information in a mutually supervised manner. The clustering results are shown in Table 6 and Table 7, where the best results are highlighted in bold. Please note that the last two rows these tables list the overall average NMI (or RI) and Std. of each adopted method across all views.

Table 6. NMI of all the adopted methods on the synthetic dataset.

Datasets	K-Means	ComBKM	Co-FKM	MVKKM	MVSpec	WV-Co-FCM	TW-K-Means	MVASM	MUC
View1-View2	0.8905 (0.0131)	0.7845 (0.0014)	0.9527 (0.0036)	0.9527 (0.0032)	0.9459 (0.0056)	0.9658 (0.0068)	0.9527 (0.0070)	0.9496 (0)	0.9784 (0.0011)
View1-View3	0.8355 (0)	0.8125 (0)	0.9355 (0)	0.9475 (0)	0.9207 (0)	0.9498 (0.0022)	0.9355 (0.0035)	0.9441 (0.0008)	0.9491 (0.0009)
View2-View3	0.8775 (0.0126)	0.8125 (0.0028)	0.9371 (0.0009)	0.9253 (0.0011)	0.9416 (0.0012)	0.9348 (0.0086)	0.9371 (0.0071)	0.9004 (0)	0.9638 (0.0012)
Avg. NMI	0.8678	0.8032	0.9418	0.9418	0.9361	0.9501	0.9418	0.9314	0.9638
Avg. Std.	0.0086	0.0014	0.0015	0.0014	0.0023	0.0058	0.0059	0.0003	0.0011

The best results are highlighted in bold.

Table 7. RI of all the adopted methods on the synthetic dataset.

Datasets	K-Means	ComBKM	Co-FKM	MVKKM	MVSpec	WV-Co-FCM	TW-K-Means	MVASM	MUC
View1-View2	0.9341 (0.0111)	0.9043 (0.0017)	0.9868 (0.0034)	0.9868 (0.0008)	0.9847 (0)	0.9574 (0.0091)	0.9868 (0.0063)	0.9583 (0)	0.9984 (0.0024)
View1-View3	0.9804 (0)	0.9419 (0.0011)	0.9804 (0)	0.9847 (0)	0.9847 (0.0004)	0.9856 (0.0011)	0.9804 (0)	0.9873 (0.0005)	0.9907 (0)
View2-View3	0.9302 (0.0110)	0.9419 (0.0021)	0.9825 (0)	0.9762 (0)	0.9762 (0.0002)	0.9337 (0.0082)	0.9825 (0.0011)	0.9475 (0)	0.9913 (0.0011)
Avg. RI	0.9482	0.9294	0.9832	0.9826	0.9819	0.9589	0.9832	0.9644	0.9935
Avg. Std.	0.0074	0.0016	0.0011	0.0003	0.0002	0.0061	0.0025	0.0002	0.0012

The best results are highlighted in bold.

From these two tables, we can see that the performances of multi-view clustering methods, i.e., Co-FKM, MVKKM, MVSpec, WV-Co-FCM TW-K-means, MVASM and MUC, are obviously superior to those of single-view equivalents, i.e., K-means and ComBKM, in terms of both NMI and RI. The reason for this is that, although single-view clustering methods are capable of clustering multi-view data using all the available features, they are not specifically designed to effectively cluster several subsets of the available features.

Another observation is that the proposed MUC method performs best among all the adopted methods in terms of overall average performance. This means that mutual supervision based on utility-based partition-level side information can effectively guide clustering for each view and improve the final multi-view clustering performance.

4.4.2. UCI Datasets

The clustering performance of the proposed method was experimentally evaluated on the seven multi-view datasets from the UCI repository. The corresponding clustering results, in terms of the mean and standard deviation of NMI and RI, are shown in Figure 5 and Figure 6. We can draw the following conclusions.

Figure 5. NMI of all the adopted methods on the seven UCI datasets.

Figure 6. RI of all the adopted methods on the seven UCI datasets.

(1)

In general, the performances of the seven multi-view methods, i.e., Co-FKM, MVKKM, MVSpec, WV-Co-FCM, TW-K-means, MVASM and MUC, are superior to those of the two single-view clustering methods, K-means and ComBKM.

(2)

The proposed method, MUC, clearly performs best on all the seven multi-view datasets in terms of overall average NMI and RI, demonstrating the benefit of multi-view clustering from mutual supervision based on utility-based partition-level side information. In terms of NMI and RI, the proposed MUC method is not always significantly superior to WV-Co-FCM or TW-K-means on every dataset (see, for example, Dermatology); however, it remains a very close second. The possible reason for this is that this dataset is so easy to cluster; as such, additional utility-based partition-level side information does not play a crucial role in enhancing the corresponding multi-view clustering performance.

(3)

An additional meaningful observation is that the weights of two views vary considerably on some datasets, such as SPECTF. This is possibly because view features are very different. That is, the amount of effective feature information provided by two views is different. Moreover, the degree of separability of the data from different views is different.

4.4.3. Real-Life Image Datasets

In this subsection, the proposed MUC is experimentally compared against other methods on the two real-life image datasets. The results for these multi-view image datasets are reported in Table 8, Table 9 and Table 10. Please note that, as with the five-view dataset, CMU PIE, we adopted an averaging strategy across whole data partitions from all the other views.

Table 8. NMI of all the adopted methods on the CMU PIE dataset.

Datasets	K-Means	ComBKM	Co-FKM	MVKKM	MVSpec	WV-Co-FCM	TW-K-Means	MVASM	MUC
D1	0.2315 (0.0196)	0.1895 (0.0175)	0.4549 (0.0081)	0.6451 (0.0008)	0.7055 (0.0014)	0.6649 (0.0136)	0.5957 (0.0103)	0.5723 (0.0058)	0.6635 (0.0061)
D2	0.2107 (0.0214)	0.2101 (0.0182)	0.4342 (0.0034)	0.5475 (0.0023)	0.6638 (0.0011)	0.6443 (0.0104)	0.6324 (0.0095)	0.5237 (0.0055)	0.6806 (0.0039)
D3	0.1804 (0.0136)	0.1943 (0.0192)	0.4693 (0.0102)	0.5985 (0.0018)	0.6720 (0.0009)	0.6805 (0.0124)	0.6483 (0.0089)	0.5724 (0.0064)	0.6823 (0.0059)
D4	0.2013 (0.0105)	0.2833 (0.0138)	0.4732 (0.0064)	0.5883 (0.0012)	0.6804 (0.0010)	0.6693 (0.0079)	0.6102 (0.0132)	0.5313 (0.0093)	0.6875 (0.0060)
D5	0.1934 (0.0103)	0.2019 (0.0157)	0.4682 (0.0087)	0.6056 (0.0006)	0.6690 (0.0013)	0.6736 (0.0131)	0.6126 (0.0146)	0.5923 (0.0066)	0.6952 (0.0053)
Avg. NMI	0.2035	0.2158	0.4600	0.5970	0.6781	0.6665	0.6198	0.5584	0.6818
Avg. Std.	0.0151	0.0169	0.0074	0.0013	0.0011	0.0115	0.0113	0.0067	0.0054

The best results are highlighted in bold.

Table 9. RI of all the adopted methods on the CMU PIE dataset.

Datasets	K-Means	ComBKM	Co-FKM	MVKKM	MVSpec	WV-Co-FCM	TW-K-Means	MVASM	MUC
D1	0.7173 (0.0113)	0.7265 (0.0173)	0.7631 (0.0024)	0.7968 (0.0019)	0.8245 (0.0014)	0.8538 (0.0092)	0.7984 (0.0075)	0.7915 (0.0037)	0.8590 (0.0036)
D2	0.7236 (0.0084)	0.7402 (0.0141)	0.7623 (0.0022)	0.7027 (0.0003)	0.8474 (0.0010)	0.8657 (0.0027)	0.7724 (0.0021)	0.7436 (0.0036)	0.8612 (0.0020)
D3	0.7224 (0.0095)	0.7237 (0.0121)	0.7386 (0.0034)	0.7522 (0.0022)	0.8760 (0.0026)	0.8553 (0.0113)	0.7992 (0.0059)	0.7981 (0.0034)	0.8709 (0.0073)
D4	0.7309 (0.0107)	0.7271 (0.0106)	0.7408 (0.0042)	0.7432 (0.0031)	0.8394 (0.0042)	0.8570 (0.0127)	0.8309 (0.0072)	0.7967 (0.0036)	0.8517 (0.0033)
D5	0.7231 (0.0137)	0.7384 (0.0083)	0.7518 (0.0054)	0.7367 (0.0017)	0.8743 (0.0018)	0.8574 (0.0071)	0.8368 (0.0053)	0.7902 (0.0052)	0.8629 (0.0052)
Avg. RI	0.7235	0.7312	0.7513	0.7463	0.8523	0.8578	0.8075	0.7840	0.8611
Avg. Std.	0.0107	0.0125	0.0035	0.0018	0.0022	0.0086	0.0056	0.0039	0.0043

The best results are highlighted in bold.

Table 10. NMI and RI of all the adopted methods on the AWA dataset.

#	K-Means	ComBKM	Co-FKM	MVKKM	MVSpec	WV-Co-FCM	TW-K-Means	MVASM	MUC
NMI	0.0902 (0.0012)	0.1084 (0.0006)	0.1450 (0.0028)	0.1155 (0.0081)	0.1087 (0.0090)	0.1041 (0)	0.1197 (0)	0.0968 (0.0058)	0.1336 (0.0029)
RI	0.6934 (0)	0.7506 (0.0010)	0.7848 (0.0061)	0.7634 (0.0022)	0.7401 (0.0031)	0.7552 (0.0015)	0.7744 (0.0012)	0.6985 (0.0041)	0.8061 (0.0032)

The best results are highlighted in bold.

According to Table 8, Table 9 and Table 10, all the multi-view clustering methods outperform the single-view equivalents, K-means and ComBKM, on two real-life image datasets. Furthermore, MUC maintains the best results in terms of the overall average NMI and RI values, owing to the use of both utility-based partition-level side information and the multi-view mutual supervision mode. Although the proposed MUC method performs slightly unsatisfactorily, and even worse in terms of RI, its clustering performance remains comparable to those of multi-view methods.

4.4.4. Statistical Analysis Results

In addition, we conducted the Friedman test [42] for statistical analysis, observing significant performance differences between the proposed MUC and other adopted methods. First, as shown in Figure 7, we calculated the average performance rankings of all the adopted methods on all datasets in terms of NMI and RI. The proposed method, MUC, clearly ranks the highest, which means that it performs best.

Figure 7. Rankings of all methods (Friedman test): (a) NMI; (b) RI.

Then, we conducted a post hoc hypothesis test to assess performance differences between MUC and other adopted methods. The obtained post hoc comparison results (${\alpha }_{Fri}=0.05$) are presented in Table 11 and Table 12, respectively, in terms of NMI and RI. Please note that we have ordered all the adopted methods (except the proposed MUC) according to the z-value obtained from the post hoc hypothesis test. The null hypothesis is clearly rejected when $p_{Fri}\le 0.006081$ (or $p_{Fri}\le 0.010781$) because $p_{Fri}\le Holm$. It can be seen from Table 11 and Table 12 that there are significant performance differences between MUC and other adopted methods in terms of NMI or RI, which indicates that our proposed method is very effective in multi-view clustering.

Table 11. Post hoc comparison of NMI for ${\alpha }_{Fri}=0.05$.

i	Methods	$\mathit{z}$	${\mathit{p}}_{\mathit{F}\mathit{r}\mathit{i}}$	Holm	Hypothesis
8	K-means	6.97137	0	0.00625	Rejected
7	ComBKM	5.9063	0	0.007143	Rejected
6	MVASM	5.099428	0	0.008333	Rejected
5	Co-FKM	3.679334	0.000234	0.01	Rejected
4	TW-K-means	3.098387	0.001946	0.0125	Rejected
3	MVKKM	2.743363	0.006081	0.016667	Rejected
2	MVSpec	2.065591	0.038867	0.025	Not Rejected
1	WV-Co-FCM	1.807392	0.070701	0.05	Not Rejected

Table 12. Post hoc comparison of RI for ${\alpha }_{Fri}=0.05$.

i	Methods	$\mathit{z}$	${\mathit{p}}_{\mathit{F}\mathit{r}\mathit{i}}$	Holm	Hypothesis
8	K-means	6.390423	0	0.00625	Rejected
7	ComBKM	5.519001	0	0.007143	Rejected
6	MVASM	4.260282	0.00002	0.008333	Rejected
5	MVKKM	3.51796	0.000435	0.01	Rejected
4	Co-FKM	3.38886	0.000702	0.0125	Rejected
3	TW-K-means	2.549714	0.010781	0.016667	Rejected
2	MVSpec	2.065591	0.038867	0.025	Not Rejected
1	WV-Co-FCM	1.936492	0.052808	0.05	Not Rejected

4.4.5. Sensitivity Analysis

We conducted experiments to study the sensitivity of MUC with respect to both the parameters $\eta$ and $\lambda$. Below we take the Multiple Features dataset as an example to carry out the sensitivity analysis. Figure 8a,b illustrate the NMI and RI changes with the increasing values of $\eta$ and $\lambda$, respectively. According to Table 3, the parameter $\eta$ is determined in $\left\{0.0001,0.001,0.01,0.1,1.0,10.0,100.0\right\}$, and the parameter $\lambda$ is determined in $\left\{{10}^{-7},{10}^{-6},\cdots {10}^0,\cdots ,{10}^6,{10}^7\right\}$. Obviously, these two parameters have considerable influence on performance of MUC. In particular, $\eta$ should keep a comparatively big value, and an appropriate value of $\lambda$ will be helpful in enhancing the performance of MUC.

Figure 8. NMI changes of MUC with different values of parameter $\eta$ and $\lambda$ on Multiple Features dataset. (a) $\eta$ (b) $\lambda$.

5. Conclusions

In this study, we propose a novel multi-view clustering method, MUC. We first designed a utility-based function to provide the proposed utility-based partition-level side information for multi-view clustering. Such information is consistent with human thinking, in that it is high-caliber and instructional for clustering. We used utility-based partition-level side information from all the other views for current view clustering. In addition, information provided by multiple other views complimented that of the current view. Then, we conducted multi-view clustering in utility-based mutual supervision mode in a K-means-like way. Moreover, we leveraged multi-view weight learning based on maximum entropy to identify the role of each view, in order to further improve multi-view clustering performance. The experimental results from various multi-view datasets indicate that the proposed MUC is comparable to—or even better than—the existing single view and multi-view clustering methods in terms of NMI and RI.

There is still a room for further study, such as the extension of both utility-based partition-level side information and the mutual supervision mode to other classical clustering methods, such as FCM. Moreover, although the regularization parameters can be determined by grid search and/or cross-validation, the corresponding procedure is still computationally expensive and even prohibitive for large scale datasets. Appropriate ways to speed up such a procedure are important and deserve in depth studies.