Integrating Dual Graph Constraints into Sparse Non-Negative Tucker Decomposition for Enhanced Co-Clustering

Abstract

Collaborative clustering is an ensemble technique that enhances clustering performance by simultaneously and synergistically processing multiple data dimensions or tasks. This is an active research area in artificial intelligence, machine learning, and data mining. A common approach to co-clustering is based on non-negative matrix factorization (NMF). While widely used, NMF-based co-clustering is limited by its bilinear nature and fails to capture the multilinear structure of data. With the objective of enhancing the effectiveness of non-negative Tucker decomposition (NTD) in image clustering tasks, in this paper, we propose a dual-graph constrained sparse non-negative Tucker decomposition NTD (GDSNTD) model for co-clustering. It integrates graph regularization, the Frobenius norm, and an $l_1$ norm constraint to simultaneously optimize the objective function. The GDSNTD mode, featuring graph regularization on both factor matrices, more effectively discovers meaningful latent structures in high-order data. The addition of the $l_1$ regularization constraint on the factor matrices may help identify the most critical original features, and the use of the Frobenius norm may produce a more highly stable and accurate solution to the optimization problem. Then, the convergence of the proposed method is proven, and the detailed derivation is provided. Finally, experimental results on public datasets demonstrate that the proposed model outperforms state-of-the-art methods in image clustering, achieving superior scores in accuracy and Normalized Mutual Information.

1. Introduction

Non-negative decomposition is a general term for a type of matrix or tensor decomposition method that requires all components in the decomposition result to be non-negative. The core idea is to decompose a non-negative data matrix or tensor into the product of several non-negative factor matrices or core tensors in order to extract interpretable latent features or components from non-negative data. The most representative non-negative decompositions are non-negative matrix factorization (NMF) and its extension in the high-dimensional domain non-negative Tucker decomposition (NTD). NMF is a common dimensionality reduction method that can extract the main features from the original data and has high interpretability [1]. When data requires more than two indices to be uniquely identified, it should be expressed in the form of a tensor. Thus, a tensor is naturally viewed as a generalization of vectors (first-order tensors) and matrices (second-order tensors) to higher orders. Non-negative tensor decomposition is a multi-dimensional data analysis method based on matrix decomposition, which can transform high-dimensional data into low-dimensional representations and retain the main features of the original data [2]. Shcherbakova  et al. investigated the advantages of non-negative Tucker decomposition [3]. Like matrix decomposition, the purpose of tensor decomposition is to extract the information or main components hidden in the original data [4,5]. The Tucker decomposition is a tensor factorization method designed for handling tensor data [6]. NTD is a concrete application of Tucker decomposition in non-negative matrix decomposition. Its core idea is to decompose a high-order non-negative tensor into the product of a core tensor and a series of factor matrices [7]. NTD not only retains the interpretability benefits of NMF but also effectively preserves the inherent multi-way structure of the original tensor data. Consequently, non-negative Tucker decomposition (NTD) is extensively utilized in a wide range of disciplines, including image analysis and text mining, to uncover the latent structures inherent in multi-way datasets.

To improve model performance, numerous regularized extensions of NMF and NTD have been developed in the recent literature. These models have demonstrated excellent performance in image clustering or data mining. Cai et al. introduced the graph-regularized non-negative matrix factorization (GNMF) algorithm, which incorporates a geometrically-based affinity graph into the NMF framework to preserve the local manifold structure of the data [8]. Sun et al. proposed graph-regularized and sparse non-negative matrix factorization with hard constraints (GSNMFC), jointly incorporating a graph regularizer and hard prior label information as well as a sparseness constraint as additional conditions to uncover the intrinsic geometrical and discriminative structures of the data space [9]. They also proposed sparse dual graph-regularized non-negative matrix factorization (SDGNMF), jointly incorporating the dual graph-regularized and sparseness constraints as additional conditions to uncover the intrinsic geometrical, discriminative structures of the data space [10]. Shang et al. proposed a novel algorithm, called graph dual regularization non-negative matrix factorization (DNMF), which simultaneously considers the geometric structures of both the data manifold and the feature manifold [11]. Long et al. proposed a novel constrained non-negative matrix factorization algorithm, called the graph-regularized discriminative non-negative matrix factorization (GDNMF), to incorporate into the NMF model both the intrinsic geometrical structure and discriminative information [12]. Saberi-Movahed et al. present a systematic analysis of NMF in dimensionality reduction, with a focus on both feature extraction and feature selection approaches [13]. Jing et al. proposed a novel semi-supervised NMF method that incorporates label regularization, basis regularization, and graph regularization [14]. Li et al. developed a manifold regularization non-negative Tucker decomposition (MR-NTD) model. To preserve the geometric information within tensor data, their method employs a manifold regularization term on the core tensor derived from the Tucker decomposition [15]. Yin and Ma incorporated this geometrically based Locally Linear Embedding (LLE) into the original NTD, thus proposing NTD-LLE for the clustering of image databases [16]. To enhance the representation learning of tensor data, Qiu et al. proposed a novel graph-regularized non-negative Tucker decomposition (GNTD) framework [17]. This method is designed to jointly extract low-dimensional parts-based representations and preserve the underlying manifold structure within the high-dimensional tensor data.

The above research demonstrates that the non-negative decomposition model significantly improves image clustering performance. However, advancements in scientific technologies have led to increasingly complex phenomena, rendering previous models inadequate for handling the resulting complexities. Building on this foundation, a common strategy to enhance image clustering accuracy has been the development of collaborative clustering (co-clustering) frameworks. This is typically achieved by incorporating specific regularization terms that capture the relationships between different data views or clusters. Co-clustering is an ensemble learning method that performs simultaneous clustering along multiple dimensions or data views [18]. When using the non-negative matrix model, the goal of co-clustering is to simultaneously identify the clusters of both the rows and columns of the two-dimensional data matrix [19]. Del Buono and Pio present a process which aims at enhancing the performance of three-factor NMF as a co-clustering method, by identifying a clearer correlation structure represented by the block matrix [20]. Deng et al. proposed graph-regularized sparse NMF (GSNMF) and graph-regularized sparse non-negative matrix tri-factorization (GSNMTF) models. By incorporating an $L_1$ norm constraint on the low-dimensional matrix, they aimed to scale the data eigenvalues and enforce sparsity. This co-clustering approach has been shown to enhance the performance of standard non-negative matrix factorization models [21]. Chachlakis, Dhanaraj, Prater-Bennette, and Markopoulos presented Dynamic $L_1$-Tucker: an algorithm for dynamic and outlier-resistant Tucker analysis of tensor data [22]. The $L_1$ norm is extensively used in convex optimization problems [23]. Ahmed et al. study a tensor-structured linear regression model over the space of sparse, low Tucker-rank tensors [24]. As deep learning models become larger and more complex, sparsity is emerging as a critical consideration for enhancing efficiency and scalability, making it a central theme in the development of new image processing and data analysis methods.

As illustrated by the above studies, building upon NMF, scholars have developed numerous models to address the evolving needs of various scientific fields. The performance of the NMF model can be significantly improved through the combined constraints of graph regularization and sparsity, which directly exploit the internal structure and inherent characteristics of the data. NTD is the extension of NMF to the high-dimensional domain. However, there are relatively few co-clustering methods for constructing high-performance NTD models that leverage the internal structure and inherent characteristics of the tensor data itself. While GNTD captures graph structure and sparse NMTF promotes sparsity, neither is designed to simultaneously learn from multiple graphs while enforcing directional sparsity patterns across different data modes. To address this gap, and inspired by advancements in co-clustering NMF, this paper proposes a GDSNTD model based on GNTD for enhanced co-clustering. The new model combines graph regularization, the Frobenius norm, and the $l_1$ norm to simultaneously optimize the objective function. In NTD, graph regularization serves to preserve the multi-linear structure of the original data. The review of the prior literature revealed that imposing multiple graph constraints on NMF models enhances their clustering performance. Motivated by this finding, we consequently introduce dual graph constraints into the NTD framework, applying them directly to the factor matrices of a tensor. This approach allows the model to capture the intrinsic data geometry more clearly, thereby improving clustering accuracy. Furthermore, we provide updated iterative optimization rules and prove the convergence of the model. Experiments on public datasets demonstrate that the proposed method outperforms several leading state-of-the-art methods.

The main contributions of this study are as follows:

• We introduce dual graph constraints into the NTD framework, applying them directly to the factor matrices of a tensor. To the best of our knowledge, no existing NTD framework integrates dual graph constraints with sparse regularization simultaneously. While graph-regularized and sparse factorization techniques exist, our model GDSNTD is the first to integrate them in a unified, constrained co-clustering optimization framework for NTD. We propose a new co-clustering version of the NTD model, equipped with three regularization terms: graph regularization, the Frobenius norm, and the $l_1$ norm. The graph regularization term captures the internal geometric structure of high-dimensional data more accurately. The $l_1$ norm term helps to scale original features in the factor matrices. The Frobenius norm improves the generalization ability of the model. Therefore, the co-clustering GDSNTD integrates strengths from graph-regularized and sparse factorization techniques so that the model yields a more accurate solution to the optimization problem.
• In the novel, unified optimization objective, we leverage the L-Lipschitz condition to derive the update rules for the proposed co-clustering GDSNTD method. Subsequently, we establish the convergence of the proposed algorithm.
• Experiments on public datasets demonstrate the effectiveness and superiority of the proposed method.

The remainder of the paper is organized as follows. In Section 2, we review the related models. In Section 3, the GDSNTD method is proposed, and its detailed inference process and the proof of convergence of the algorithm are illustrated. Section 4 presents the performance of the proposed model via experiments on various datasets. Finally, we present our conclusions in Section 5 and outline future work in Section 6.

2. Preliminaries

In this section, we review the related models.

(A)

NTD

  Given a non-negative data tensor $\mathcal{X}\in {\mathbb{R}}_{+}^{I_1\times I_2\times I_3}$, non-negative tucker decomposition (NTD) aims at decomposing the non-negative tensor $\mathcal{X}$ into a non-negative core tensor $\mathcal{G}\in {\mathbb{R}}_{+}^{I_1\times I_2\times I_3}$ multiplied by N non-negative factor matrices $\mathbf{U}\in {\mathbb{R}}_{+}^{I_1\times J_1}$, $\mathbf{V}\in {\mathbb{R}}_{+}^{I_2\times J_2}$, and $\mathbf{W}\in {\mathbb{R}}_{+}^{I_3\times J_3}$ along each mode [7]. NTD minimizes the sum of squared residues between the data tensor $\mathcal{X}$ and the multi-linear product of core tensor $\mathcal{G}$ and factor matrices $\mathbf{U}$, $\mathbf{V}$, and $\mathbf{W}$, which is expressed as

$$F_{NTD}=\underset{\mathcal{G}⩾0,\mathbf{U},\mathbf{V},\mathbf{W}⩾0}{min}{∥\begin{array}{c}\mathcal{X}-\mathcal{G}{\times }_1\mathbf{U}{\times }_2\mathbf{V}{\times }_3\mathbf{W}\end{array}∥}_F^2.$$

  NTD provides an effective embedding and representation for image tensor data $\mathcal{X}$. However, NTD does not consider the geometrical structure of image tensor data $\mathcal{X}$.

(B)

GNTD

  Qiu et al. [17] proposed the GNTD model, which incorporates the graph regularization term into the original NTD method. The GNTD is mathematically formulated as

$$F_{GNTD}=\underset{\mathcal{G}⩾0,\mathbf{U},\mathbf{V},\mathbf{W}⩾0}{min}{∥\begin{array}{c}\mathcal{X}-\mathcal{G}{\times }_1\mathbf{U}{\times }_2\mathbf{V}{\times }_3\mathbf{W}\end{array}∥}_F^2+\lambda Tr\left({\mathbf{W}}^{\top }\mathbf{L}\mathbf{W}\right),$$

where $\lambda ⩾0$ is the regularization parameter for balancing the importance of the graph regularization term and the reconstruction error term, tr(·) denotes the trace of the matrix, and $\mathbf{L}$ is a Laplacian matrix which characterizes the data manifold. $\mathbf{L}$ is defined as $\mathbf{L}=\mathbf{H}-\mathbf{A}$, and ${\left(\mathbf{H}\right)}_{ii}={\sum }_j{\left(\mathbf{A}\right)}_{ij}$. $\mathbf{A}$ is the adjacency matrix of a p-nearest neighbor graph regarding the data points $({\mathbf{Y}}_1,\dots ,{\mathbf{Y}}_2,\dots )$ constructed using the $0-1$ scheme.

$${\left(\mathbf{A}\right)}_{ij}=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}{\mathbf{Y}}_i\in N_p\left({\mathbf{Y}}_j\right)\mathrm{and}{\mathbf{Y}}_j\in N_p\left({\mathbf{Y}}_i\right),\hfill \\ 0,\hfill & \mathrm{otherwise},\hfill \end{array}\right.$$

where $N_p\left({\mathbf{Y}}_j\right)$ represents the set of p-nearest neighbor points of $\left({\mathbf{Y}}_j\right)$.

  The GNTD algorithm ingeniously combines graph constraints with non-negative constraints. This approach effectively preserves the local geometric structure of the data, enabling the learned features to better capture its intrinsic structure. As a result, GNTD significantly enhances the performance of traditional tensor decomposition models and has become a powerful tool for handling complex high-dimensional data, receiving considerable attention in machine learning and data mining.

(C)

GSNMTF

  The graph-regularized sparse non-negative matrix tri-factorization (GSNMTF) model incorporates graph regularization and the $l_1$ norm into the standard NMF objective function [21]. This integration not only preserves the geometric structure of the data and feature spaces but also promotes sparsity in the resulting factor matrices. The objective function of GSNMTF is defined as

$$F_{GSNMTF}=\underset{\mathbf{U}⩾0,\mathbf{S}⩾0,\mathbf{V}⩾0}{min}{∥\begin{array}{c}\mathbf{X}-\mathbf{U}\mathbf{S}{\mathbf{V}}^{\top }\end{array}∥}_F^2+{\lambda }_{U}Tr\left({\mathbf{U}}^{\top }{\mathbf{L}}_{\mathbf{U}}\mathbf{U}\right)+{\lambda }_{V}Tr\left({\mathbf{V}}^{\top }{\mathbf{L}}_{\mathbf{V}}\mathbf{V}\right)+$$

$${\gamma }_U{∥\begin{array}{c}\mathbf{U}\end{array}∥}_1^1+{\gamma }_V{∥\begin{array}{c}\mathbf{V}\end{array}∥}_1^1,$$

where tr(·) is the trace of the matrix, and ${\gamma }_U$ and ${\gamma }_V$ are the parameters for controlling the sparseness of the matrices $\mathbf{U}$ and $\mathbf{V}$, respectively. ${∥\begin{array}{c}·\end{array}∥}_F$ and ${∥\begin{array}{c}·\end{array}∥}_1$ are the Frobenius norm and $l_1$ norm, respectively.

3. Dual Graph Constraints into Sparse Non-Negative Tucker Decomposition (GDSNTD) Model

This section introduces the proposed GDSNTD model. We begin with the problem setup. Given a third-order tensor $\mathcal{X}\in {\mathbb{R}}_{+}^{I_1\times I_2\times I_3}$, where $I_1$, $I_2$, and $I_3$ are the numbers of rows, columns, and tubes of tensor, respectively. The Tucker decomposition of the tensor is given by

$$\begin{array}{c}\hfill \mathcal{X}\approx \mathcal{G}{\times }_1\mathbf{U}{\times }_2\mathbf{V}{\times }_3\mathbf{W}.\end{array}$$

(1)

It decomposes a tensor into a core tensor multiplied by a matrix along each mode. Here the tensor $\mathcal{G}\in {\mathbb{R}}_{+}^{J_1\times J_2\times J_3}$ is called the core tensor and its entries show the level of interaction between the different components. $\mathbf{U}\in {\mathbb{R}}_{+}^{I_1\times J_1}$, $\mathbf{V}\in {\mathbb{R}}_{+}^{I_2\times J_2}$, and $\mathbf{W}\in {\mathbb{R}}_{+}^{I_3\times J_3}$ are the factor matrices (which are usually orthogonal) and can be thought of as the principal components in each mode, and $\{J_1,J_2,J_3\}\ll min\{I_1,I_2,I_3\}$.

3.1. Data and Feature Graphs

Many research studies have demonstrated that geometric structure not only affects the data space but also the feature space and that high-dimensional data are usually located in a low-dimensional sub-manifold of the ambient space, and this underlying geometrical information of data could be obtained by modeling a neighbor graph. We impose a graph regularization constraint on the factor matrix $\mathbf{V}$. We construct a p-nearest neighbors data space, whose vertices are $\{{\mathcal{X}}_{:,:,1},\dots ,{\mathcal{X}}_{:,:,N}\}$. Then, we encode their geometrical information by connecting each tensor subject with its p-nearest neighbors, thereby constructing a p-nearest neighbor graph [8] with a binary $(0-1)$ weighting scheme. The weight matrix is defined as

$${\left[{\mathbf{C}}_{\mathbf{V}}\right]}_{ijk}=\left\{\begin{array}{cc}1\hfill & \mathrm{if}{\mathcal{X}}_{j,:,k}\in {\mathcal{N}}_p\left({\mathcal{X}}_{i,:,k}\right)\mathrm{and}{\mathcal{X}}_{i,:,k}\in N_p\left({\mathcal{X}}_{j,:,k}\right),\hfill \\ 0\hfill & \mathrm{otherwise},\hfill \end{array}\right.$$

where ${\mathcal{N}}_p\left({\mathcal{X}}_{j,:,k}\right)$ represents the set of p samples closest to ${\mathcal{X}}_{i,:,k}$ in the graph. The graph Laplacian of the data graph is defined as ${\mathbf{L}}_{\mathbf{V}}={\mathbf{D}}_{\mathbf{V}}-{\mathbf{C}}_{\mathbf{V}}$, where ${\mathbf{D}}_{\mathbf{V}}$ is the diagonal degree matrix whose elements are given by ${\left[{\mathbf{D}}_{\mathbf{V}}\right]}_{ii}={\sum }_j{\left[{\mathbf{C}}_{\mathbf{V}}\right]}_{ij}$.

Similarly, we impose a graph regularization constraint on the factor matrix $\mathbf{W}$. We can also construct a p-nearest neighbors graph of the feature space and define the weight matrix as follows:

$${\left[{\mathbf{C}}_{\mathbf{W}}\right]}_{ijk}=\left\{\begin{array}{cc}1\hfill & \mathrm{if}{\mathcal{X}}_{k,j,:}\in {\mathcal{N}}_p\left({\mathcal{X}}_{i,j,:}\right)\mathrm{and}{\mathcal{X}}_{i,j,:}\in N_p\left({\mathcal{X}}_{k,j,:}\right),\hfill \\ 0\hfill & \mathrm{otherwise},\hfill \end{array}\right.$$

where ${\mathcal{N}}_p\left({\mathcal{X}}_{k,j,:}\right)$ represents the set of p samples closest to ${\mathcal{X}}_{i,j,:}$ in the graph. The graph Laplacian of the data graph is defined as ${\mathbf{L}}_{\mathbf{W}}={\mathbf{D}}_{\mathbf{W}}-{\mathbf{C}}_{\mathbf{W}}$.

3.2. Objective Function of GDSNTD

In this part, we present our objective function of GDSNTD, which is designed to approximate the original high-dimensional tensor by yielding cleaner and sparser low-dimensional matrices. This model is able to present the geometric structure of tensors more clearly than GNTD and makes up for the deficiency of GSNMTF in handling high-dimensional data. The proposed model has advanced the development of the NTD framework by improving its convergence properties and has facilitated its application in complex tasks such as image clustering analysis. It integrates graph regularization, the Frobenius norm, and a sparsity constraint, and its objective function is defined as

$$\begin{array}{cc}\hfill F_{GDSNTD}=& \underset{\mathcal{G}⩾0,\mathbf{U},\mathbf{V},\mathbf{W}⩾0}{min}{∥\begin{array}{c}\mathcal{X}-\mathcal{G}{\times }_1\mathbf{U}{\times }_2\mathbf{V}{\times }_3\mathbf{W}\end{array}∥}_F^2+{\alpha }_V{∥\begin{array}{c}\mathbf{V}\end{array}∥}_F^2+{\alpha }_W{∥\begin{array}{c}\mathbf{W}\end{array}∥}_F^2+\hfill \\ & {\lambda }_{V}Tr\left({\mathbf{V}}^{\top }{\mathbf{L}}_{\mathbf{V}}\mathbf{V}\right)+{\lambda }_{W}Tr\left({\mathbf{W}}^{\top }{\mathbf{L}}_{\mathbf{W}}\mathbf{W}\right)+{\gamma }_V{∥\begin{array}{c}\mathbf{V}\end{array}∥}_1^1+{\gamma }_W{∥\begin{array}{c}\mathbf{W}\end{array}∥}_1^1,\hfill \end{array}$$

(2)

where tr(·) is the trace of the matrix. ${\alpha }_V$ is the parameter of $\mathbf{V}$, ${\alpha }_W$ is the parameter of $\mathbf{W}$. ${\lambda }_V$ and ${\lambda }_W$ are the graph regularization parameters of $\mathbf{V}$ and $\mathbf{W}$, respectively. They govern the trade-off between preserving the intrinsic graph structure of the data and optimizing other objective terms. ${\gamma }_V$ and ${\gamma }_W$ are coefficients of the $l_1$ norm. They serve as a trade-off between the sparsity of the factor matrices and other objective terms. It is worth noting that the norms in the objective function are entrywise norms [25]. ${∥\begin{array}{c}·\end{array}∥}_F$ and ${∥\begin{array}{c}·\end{array}∥}_1$ represent the Frobenius norm and $l_1$ norm, respectively.

Equivalently, Equation (2) is to be rewritten in matrix form as

$$\begin{array}{cc}\hfill F_{GDSNTD}=& \underset{\mathcal{G}⩾0,\mathbf{U},\mathbf{V},\mathbf{W}⩾0}{min}{∥\begin{array}{c}{\mathbf{X}}_{\left(1\right)}-\mathbf{U}{\mathbf{G}}_{\left(1\right)}{(\mathbf{W}\otimes \mathbf{V})}^{\top }\end{array}∥}_F^2+{\alpha }_V{∥\begin{array}{c}\mathbf{V}\end{array}∥}_F^2+{\alpha }_W{∥\begin{array}{c}\mathbf{W}\end{array}∥}_F^2+\hfill \\ & {\lambda }_{V}Tr\left({\mathbf{V}}^{\top }{\mathbf{L}}_{\mathbf{V}}\mathbf{V}\right)+{\lambda }_{W}Tr\left({\mathbf{W}}^{\top }{\mathbf{L}}_{\mathbf{W}}\mathbf{W}\right)+{\gamma }_V{∥\begin{array}{c}\mathbf{V}\end{array}∥}_1^1+{\gamma }_W{∥\begin{array}{c}\mathbf{W}\end{array}∥}_1^1,\hfill \end{array}$$

(3)

where the tensor $\mathcal{X}$ is expanded according to mode-1. When the tensor $\mathcal{X}$ is expanded according to mode-2 or mode-3, it is to be rewritten in the following forms as Equations (4) and Equation (5), respectively:  

$$\begin{array}{cc}\hfill F_{GDSNTD}=& \underset{\mathcal{G}⩾0,\mathbf{U},\mathbf{V},\mathbf{W}⩾0}{min}{∥\begin{array}{c}{\mathbf{X}}_{\left(2\right)}-\mathbf{V}{\mathbf{G}}_{\left(2\right)}{(\mathbf{W}\otimes \mathbf{U})}^{\top }\end{array}∥}_F^2+{\alpha }_V{∥\begin{array}{c}\mathbf{V}\end{array}∥}_F^2+{\alpha }_W{∥\begin{array}{c}\mathbf{W}\end{array}∥}_F^2+\hfill \\ & {\lambda }_{V}Tr\left({\mathbf{V}}^{\top }{\mathbf{L}}_{\mathbf{V}}\mathbf{V}\right)+{\lambda }_{W}Tr\left({\mathbf{W}}^{\top }{\mathbf{L}}_{\mathbf{W}}\mathbf{W}\right)+{\gamma }_V{∥\begin{array}{c}\mathbf{V}\end{array}∥}_1^1+{\gamma }_W{∥\begin{array}{c}\mathbf{W}\end{array}∥}_1^1,\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill F_{GDSNTD}=& \underset{\mathcal{G}⩾0,\mathbf{U},\mathbf{V},\mathbf{W}⩾0}{min}{∥\begin{array}{c}{\mathbf{X}}_{\left(3\right)}-\mathbf{W}{\mathbf{G}}_{\left(3\right)}{(\mathbf{V}\otimes \mathbf{U})}^{\top }\end{array}∥}_F^2+{\alpha }_V{∥\begin{array}{c}\mathbf{V}\end{array}∥}_F^2+{\alpha }_W{∥\begin{array}{c}\mathbf{W}\end{array}∥}_F^2+\hfill \\ & {\lambda }_{V}Tr\left({\mathbf{V}}^{\top }{\mathbf{L}}_{\mathbf{V}}\mathbf{V}\right)+{\lambda }_{W}Tr\left({\mathbf{W}}^{\top }{\mathbf{L}}_{\mathbf{W}}\mathbf{W}\right)+{\gamma }_V{∥\begin{array}{c}\mathbf{V}\end{array}∥}_1^1+{\gamma }_W{∥\begin{array}{c}\mathbf{W}\end{array}∥}_1^1.\hfill \end{array}$$

(5)

3.3. Inference of GDSNTD

In this section, the detailed inference is illustrated, and then, a corresponding algorithm is designed.

3.3.1. Fix $\mathbf{U}\in {\mathbb{R}}_{+}^{I_1\times J_1}$, $\mathbf{V}\in {\mathbb{R}}_{+}^{I_2\times J_2}$, $\mathbf{W}\in {\mathbb{R}}_{+}^{I_3\times J_3}$, Solve $\mathcal{G}$

Letting ${\lambda }_{G}G$ be the Lagrange multiplier [26] for constraint $\mathcal{G}⩾0$, and our objective function is written in matrix form as

$$\widehat{\mathcal{L}}\left(\mathcal{G}\right)={∥\begin{array}{c}vec\left(\mathcal{X}\right)-(\mathbf{W}\otimes \mathbf{V}\otimes \mathbf{U})vec\left(\mathcal{G}\right)\end{array}∥}_F^2+vec{\left(\mathcal{G}\right)}^{\top }vec\left({\lambda }_G\right).$$

(6)

To obtain the optimal solution, the gradient descent algorithm [27] is used to solve Equation (6). The gradient is as follows:

$$\begin{array}{c}\hfill {\displaystyle \frac{\partial \widehat{\mathcal{L}}}{\partial vec\left(\mathcal{G}\right)}}=2{(\mathbf{W}\otimes \mathbf{V}\otimes \mathbf{U})}^{\top }(\mathbf{W}\otimes \mathbf{V}\otimes \mathbf{U})vec\left(\mathcal{G}\right)-2{(\mathbf{W}\otimes \mathbf{V}\otimes \mathbf{U})}^{\top }vec\left(\mathcal{X}\right)+vec\left({\lambda }_G\right).\end{array}$$

(7)

Let ${\mathcal{G}}^{(}t)$ represent the corresponding value during the tth round of updates. Therefore, using the Karush–Kuhn–Tucker (KKT) condition [27], ${\displaystyle \frac{\partial \mathcal{L}}{\partial vec\left(\mathcal{G}\right)}}=0$ and $vec{\left(\mathcal{G}\right)}_{i}vec{\left({\lambda }_{\mathcal{G}}\right)}_i=0$, we obtain the following updating rule:

$${\left(vec\left(\mathcal{G}\right)\right)}_i\leftarrow {\left(vec\left(\mathcal{G}\right)\right)}_i{\displaystyle \frac{{\left({(\mathbf{W}\otimes \mathbf{V}\otimes \mathbf{U})}^{\top }vec\left(\mathcal{X}\right)\right)}_i}{{\left({(\mathbf{W}\otimes \mathbf{V}\otimes \mathbf{U})}^{\top }(\mathbf{W}\otimes \mathbf{V}\otimes \mathbf{U})vec\left(\mathcal{G}\right)\right)}_i}}.$$

(8)

3.3.2. Fix $\mathcal{G}$, $\mathbf{V}\in {\mathbf{R}}_{+}^{I_2\times J_2}$ and $\mathbf{W}\in {\mathbf{R}}_{+}^{I_3\times J_3}$, Solve $\mathbf{U}\in {\mathbf{R}}_{+}^{I_1\times J_1}$

At this stage, similar to the update of $\mathcal{G}$, letting ${\lambda }_{\mathbf{U}}\mathbf{U}$ be the Lagrange multiplier for constraint $\mathbf{U}⩾0$, our optimization goal is to be written in matrix form as follows:

$$\begin{array}{c}\hfill \widehat{\mathcal{O}}\left(\mathbf{U}\right)={∥\begin{array}{c}{\mathbf{X}}_{\left(1\right)}-\mathbf{U}{\mathbf{G}}_{\left(1\right)}{(\mathbf{W}\otimes \mathbf{V})}^{\top }\end{array}∥}_F^2+Tr\left({\mathbf{U}}^{\top }{\lambda }_{\mathbf{U}}\right).\end{array}$$

(9)

The function (10) is to be rewritten as

$$\begin{array}{cc}\hfill \widehat{\mathcal{O}}\left(\mathbf{U}\right)=& Tr\left({\mathbf{X}}_{\left(1\right)}{\mathbf{X}}_{\left(1\right)}^{\top }\right)-2Tr\left({\mathbf{X}}_{\left(1\right)}(\mathbf{W}\otimes \mathbf{V}){\mathbf{G}}_{\left(1\right)}^{\top }{\mathbf{U}}^{\top }\right)+\hfill \\ & Tr\left(\mathbf{U}{\mathbf{G}}_{\left(1\right)}\left(\left({\mathbf{W}\otimes \mathbf{V}}^{\top }\right)(\mathbf{W}\otimes \mathbf{V})\right){\mathbf{G}}_{\left(1\right)}^{\top }{\mathbf{U}}^{\top }\right)+\sum _{r=1}^{N}Tr\left({\mathbf{U}}^{\top }{\lambda }_{\mathbf{U}}\right).\hfill \end{array}$$

(10)

Now, the partial derivative of $\widehat{\mathcal{O}}\left(\mathbf{U}\right)$ to $\mathbf{U}$ is

$$\begin{array}{c}\hfill {\displaystyle \frac{\partial \widehat{\mathcal{O}}\left(\mathbf{U}\right)}{\partial \left(\mathbf{U}\right)}}=-2{\mathbf{X}}_{\left(1\right)}(\mathbf{W}\otimes \mathbf{V}){\mathbf{G}}_{\left(1\right)}^{\top }+2\mathbf{U}{\mathbf{G}}_{\left(1\right)}\left({(\mathbf{W}\otimes \mathbf{V})}^{\top }(\mathbf{W}\otimes \mathbf{V})\right){\mathbf{G}}_{\left(1\right)}^{\top }+{\lambda }_{\mathbf{U}}.\end{array}$$

(11)

According to the (Karush–Kuhn–Tucker) (KKT) condition [27], ${\displaystyle \frac{\partial \widehat{\mathcal{O}}\left(\mathbf{U}\right)}{\partial \left(\mathbf{U}\right)}}=0$, so

$${\lambda }_{\mathbf{U}}=2{\mathbf{X}}_{\left(1\right)}(\mathbf{W}\otimes \mathbf{V}){\mathbf{G}}_{\left(1\right)}^{\top }-2\mathbf{U}{\mathbf{G}}_{\left(r\right)}\left({\left(\mathbf{W}\otimes \mathbf{V}\right)}^{\top }(\mathbf{W}\otimes \mathbf{V})\right){\mathbf{G}}_{\left(r\right)}^{\top }.$$

And also because ${\lambda }_{\mathbf{U}}\odot \mathbf{U}=0$, ⊙ denotes the Hadamard product, and we can obtain the following:

$$(2{\mathbf{X}}_{\left(1\right)}(\mathbf{W}\otimes \mathbf{V}){\mathbf{G}}_{\left(1\right)}^{\top }-2\mathbf{U}{\mathbf{G}}_{\left(1\right)}\left({\left(\mathbf{W}\otimes \mathbf{V}\right)}^{\top }(\mathbf{W}\otimes \mathbf{V})\right){\mathbf{G}}_{\left(1\right)}^{\top })\odot \mathbf{U}=0.$$

(12)

Therefore, the update rule of $\mathbf{U}$ is

$${\mathbf{U}}_{ij}\leftarrow {\mathbf{U}}_{ij}{\displaystyle \frac{{\left({\mathbf{X}}_{\left(1\right)}(\mathbf{W}\otimes \mathbf{V}){\mathbf{G}}_{\left(1\right)}^{\top }\right)}_{ij}}{{\left(\mathbf{U}{\mathbf{G}}_{\left(1\right)}\left({\left(\mathbf{W}\otimes \mathbf{V}\right)}^{\top }(\mathbf{W}\otimes \mathbf{V})\right){\mathbf{G}}_{\left(1\right)}^{\top }\right)}_{ij}}}.$$

(13)

3.3.3. Fix $\mathcal{G}$, $\mathbf{U}\in {\mathbb{R}}_{+}^{I_1\times J_1}$ and $\mathbf{W}\in {\mathbb{R}}_{+}^{I_3\times J_3}$, Solve $\mathbf{V}\in {\mathbb{R}}_{+}^{I_2\times J_2}$

At this stage, our goal is to minimize the function $\mathcal{P}\left(\mathbf{V}\right)$. The optimization goal in (4) is to be defined as

$$\mathcal{P}\left(\mathbf{V}\right)=\widehat{\mathcal{P}}\left(\mathbf{V}\right)+{\gamma }_V{∥\begin{array}{c}\mathbf{V}\end{array}∥}_1^1,$$

(14)

where

$$\begin{array}{c}\hfill \widehat{\mathcal{P}}\left(\mathbf{V}\right)={∥\begin{array}{c}{\mathbf{X}}_{\left(2\right)}-\mathbf{V}{\mathbf{G}}_{\left(2\right)}{(\mathbf{W}\otimes \mathbf{U})}^{\top }\end{array}∥}_F^2+{\alpha }_V{∥\begin{array}{c}\mathbf{V}\end{array}∥}_F^2+{\lambda }_{V}Tr\left({\mathbf{V}}^{\top }{\mathbf{L}}_{\mathbf{V}}\mathbf{V}\right)+Tr\left({\lambda }_{\mathbf{V}}\right).\end{array}$$

(15)

Since $\widehat{\mathcal{P}}\left({\mathbf{V}}^{\prime }\right)$ is a convex function with respect to ${\mathbf{V}}^{\prime }$ and it satisfies the L-Lipschitz condition [28] as shown in Equation (14), there exists a constant $L_1⩾0$ such that

$$∥\begin{array}{c}\nabla \widehat{\mathcal{P}}\left({\mathbf{V}}^{\prime }\right)-\nabla \widehat{\mathcal{P}}\left(\mathbf{V}\right)\end{array}∥⩽{\mathbf{L}}_1∥\begin{array}{c}{\mathbf{V}}^{\prime }-\mathbf{V}\end{array}∥,\forall ({\mathbf{V}}^{\prime },\mathbf{V}),$$

(16)

where $\widehat{\mathcal{P}}\left({\mathbf{V}}^{\prime }\right)$ is to be by the approximated second-order Taylor expansion near $\mathbf{V}$ as

$$\begin{array}{cc}\hfill \tilde{\mathcal{P}}\left(\mathbf{V}\right)\approx & \widehat{\mathcal{P}}\left({\mathbf{V}}^{\prime }\right)+{\displaystyle \frac{L_1}{2}}{(\mathbf{V}-{\mathbf{V}}^{\prime })}^2+{\displaystyle \frac{\partial \widehat{\mathcal{P}}({\mathbf{V}}^{\prime })}{\partial ({\mathbf{V}}^{\prime })}}(\mathbf{V}-{\mathbf{V}}^{\prime })\hfill \\ & ={\displaystyle \frac{L_1}{2}}{(\mathbf{V}-({\mathbf{V}}^{\prime }-{\displaystyle \frac{1}{L_1}}{\displaystyle \frac{\partial \widehat{\mathcal{P}}({\mathbf{V}}^{\prime })}{\partial {\mathbf{V}}^{\prime }}}))}^2+\Phi ({\mathbf{V}}^{\prime }),\hfill \\ & \Phi ({\mathbf{V}}^{\prime })=\widehat{\mathcal{P}}({\mathbf{V}}^{\prime })-{\displaystyle \frac{1}{2L_1}}{({\displaystyle \frac{\partial \widehat{\mathcal{P}}({\mathbf{V}}^{\prime })}{\partial {\mathbf{V}}^{\prime }}})}^2.\hfill \end{array}$$

(17)

Let ${\mathbf{V}}^{\prime }={\mathbf{V}}^{\left(t\right)}$, the minimum value of Equation (17) is then obtained as ${\mathbf{V}}^{(t+1)}$ and

$${\mathbf{V}}^{(t+1)}={\mathbf{V}}^{\left(t\right)}-{\displaystyle \frac{1}{L_1}}{\displaystyle \frac{\partial \widehat{\mathcal{P}}\left({\mathbf{V}}^{\left(t\right)}\right)}{\partial \left({\mathbf{V}}^{\left(t\right)}\right)}}.$$

(18)

Since $\Phi \left({\mathbf{V}}^{\left(t\right)}\right)$ is independent of the optimization variable $\mathbf{V}$, it can be treated as a constant. Consequently, the objective function for updating $\mathbf{V}$ simplifies to

$$\begin{array}{c}\hfill \mathbf{V}(t+1)=arg\underset{\mathbf{V}}{min}[{\displaystyle \frac{L_1}{2}}{∥\begin{array}{c}\mathbf{V}-({\mathbf{V}}^{\left(t\right)}-{\displaystyle \frac{1}{L_1}}{\displaystyle \frac{\partial \widehat{\mathcal{P}}\left({\mathbf{V}}^{\left(t\right)}\right)}{\partial {\mathbf{V}}^{\left(t\right)}}})\end{array}∥}_2^2+{\gamma }_V{∥\begin{array}{c}\mathbf{V}\end{array}∥}_1^1].\end{array}$$

(19)

Let ${\mathbf{S}}^{\left(t\right)}={\mathbf{V}}^{\left(t\right)}-{\displaystyle \frac{1}{L_1}}{\displaystyle \frac{\partial \widehat{\mathcal{P}}\left({\mathbf{V}}^t\right)}{\partial {\mathbf{V}}^{\left(t\right)}}}$, then, for solving Equation (19), we first solve ${\mathbf{S}}^{\left(t\right)}$ and then

$$\begin{array}{cc}\hfill {\mathbf{V}}^{(t+1)}=& arg\underset{\mathbf{V}}{min}[{\displaystyle \frac{L_1}{2}}{∥\begin{array}{c}\mathbf{V}-{\mathbf{S}}^{\left(t\right)}\end{array}∥}_2^2+{\gamma }_V{∥\begin{array}{c}\mathbf{V}\end{array}∥}_1^1].\hfill \end{array}$$

(20)

Now, the partial derivative of $\widehat{\mathcal{P}}\left({\mathbf{V}}^{\left(t\right)}\right)$ to ${\mathbf{V}}^{\left(t\right)}$ is  

$$\begin{array}{cc}\hfill {\displaystyle \frac{\partial \widehat{\mathcal{P}}\left({\mathbf{V}}^{\left(t\right)}\right)}{\partial \left({\mathbf{V}}^{\left(t\right)}\right)}}=& -2({\mathbf{X}}_{\left(2\right)}^{\left(t\right)}-{\mathbf{V}}^{\left(t\right)}{\mathbf{G}}_{\left(2\right)}^{\left(t\right)}{({\mathbf{W}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)})}^{\top }){\left({\mathbf{G}}_{\left(2\right)}^{\left(t\right)}{({\mathbf{W}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)})}^{\top }\right)}^{\top }+\hfill \\ & 2{\alpha }_V{\mathbf{V}}^{\left(t\right)}+2{\lambda }_V{\mathbf{L}}_{\mathbf{V}}^{\left(t\right)}{\mathbf{V}}^{\left(t\right)}+{\lambda }_{{\mathbf{V}}^{\left(t\right)}}\hfill \\ & =-2{\mathbf{X}}_{\left(2\right)}^{\left(t\right)}({\mathbf{W}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)}){{\mathbf{G}}_{\left(2\right)}^{\left(t\right)}}^{\top }+\hfill \\ & 2{\mathbf{V}}^{\left(t\right)}{\mathbf{G}}_{\left(2\right)}^{\left(t\right)}{({\mathbf{W}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)})}^{\top }({\mathbf{W}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)}){{\mathbf{G}}_{\left(2\right)}^{\left(t\right)}}^{\top }+\hfill \\ & 2{\alpha }_V{\mathbf{V}}^{\left(t\right)}+2{\lambda }_V({\mathbf{D}}_{\mathbf{V}}^{\left(t\right)}-{\mathbf{C}}_{\mathbf{V}}^{\left(t\right)}){\mathbf{V}}^{\left(t\right)}+{\lambda }_{\mathbf{V}}^{\left(t\right)}.\hfill \end{array}$$

(21)

where ${\mathbf{D}}_{\mathbf{V}}^{\left(t\right)}$ and ${\mathbf{C}}_{\mathbf{V}}^{\left(t\right)}$ should be solved at the same time. According to the (Karush–Kuhn–Tucker) (KKT) condition [27], ${\lambda }_{\mathbf{V}}{\mathbf{V}}^{\left(t\right)}=0$, and  ${\displaystyle \frac{\partial \widehat{\mathcal{P}}\left({\mathbf{V}}^{\left(t\right)}\right)}{\partial {\mathbf{V}}^{\left(t\right)}}}=0$, we can obtain the following:

$$\begin{array}{cc}\hfill & (-2{\mathbf{X}}_{\left(2\right)}^{\left(t\right)}({\mathbf{W}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)}){{\mathbf{G}}_{\left(2\right)}^{\left(t\right)}}^{\top }-2{\lambda }_V{\mathbf{C}}_{\mathbf{V}}^{\left(t\right)}{\mathbf{V}}^{\left(t\right)}+\hfill \\ & 2{\mathbf{V}}^{\left(t\right)}{\mathbf{G}}_{\left(2\right)}^{\left(t\right)}{({\mathbf{W}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)})}^T({\mathbf{W}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)}){{\mathbf{G}}_{\left(\mathbf{2}\right)}^{\left(\mathbf{t}\right)}}^{\top }+2{\alpha }_V{\mathbf{V}}^{\left(t\right)}+\hfill \\ & 2{\lambda }_V{\mathbf{D}}_{\mathbf{V}}^{\left(t\right)}{\mathbf{V}}^{\left(t\right)}){\mathbf{V}}^{\left(t\right)}=0.\hfill \end{array}$$

(22)

Therefore, the update rule of ${\mathbf{S}}^{\left(t\right)}$ is

$$\begin{array}{c}\hfill {\mathbf{S}}_{ij}^{\left(t\right)}={{\mathbf{V}}_{ij}}^{\left(t\right)}{\displaystyle \frac{{[{{\mathbf{X}}_{\left(2\right)}}^{\left(t\right)}({\mathbf{W}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)}){{\mathbf{G}}_{\left(2\right)}^{\left(t\right)}}^{\top }+{\lambda }_V{\mathbf{C}}_{\mathbf{V}}^{\left(t\right)}{\mathbf{V}}^{\left(t\right)}]}_{ij}}{{[{\mathbf{V}}^{\left(t\right)}{\mathbf{G}}_{\left(2\right)}^{\left(t\right)}{({\mathbf{W}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)})}^{\top }({\mathbf{W}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)}){{\mathbf{G}}_{\left(2\right)}^{\left(t\right)}}^{\top }+{\alpha }_V{\mathbf{V}}^{\left(t\right)}+{\lambda }_V{\mathbf{D}}_{\mathbf{V}}^{\left(t\right)}{\mathbf{V}}^{\left(t\right)}]}_{ij}}}.\end{array}$$

(23)

To solve the objective function in Equation (19), we minimize the ${\mathbf{S}}_{ij}^{\left(t\right)}$ term using the gradient descent and apply a soft threshold [28] to minimize the ${∥\begin{array}{c}\mathbf{V}\end{array}∥}_1^1$ regularizer. Consequently, the update rule for $\mathbf{V}$ is derived as follows:

$${{\mathbf{V}}_{ij}}^{(t+1)}=\left\{\begin{array}{cc}{\mathbf{S}}_{ij}^{\left(t\right)}-{\displaystyle \frac{{\gamma }_V}{L_1}},\hfill & {\mathbf{S}}_{ij}^{\left(t\right)}>{\displaystyle \frac{{\gamma }_V}{L_1}},\hfill \\ 0,\hfill & {\mathbf{S}}_{ij}^{\left(t\right)}\le {\displaystyle \frac{{\gamma }_V}{L_1}}.\hfill \end{array}\right.$$

(24)

3.3.4. Fix $\mathcal{G}$, $\mathbf{U}\in {\mathbb{R}}_{+}^{I_1\times J_1}$ and $\mathbf{V}\in {\mathbb{R}}_{+}^{I_2\times J_2}$, Solve $\mathbf{W}\in {\mathbb{R}}_{+}^{I_3\times J_3}$

Similarly, to update the factor matrix $\mathbf{W}$, we write the objective function $\mathcal{Q}\left(\mathbf{W}\right)$ as follows:

$$\mathcal{Q}\left(\mathbf{W}\right)=\widehat{\mathcal{Q}}\left(\mathbf{W}\right)+{\gamma }_W{∥\begin{array}{c}\mathbf{W}\end{array}∥}_1^1,$$

(25)

where

$$\begin{array}{c}\hfill \widehat{\mathcal{Q}}\left(\mathbf{W}\right)={∥\begin{array}{c}{\mathbf{X}}_{\left(3\right)}-\mathbf{W}{\mathbf{G}}_{\left(3\right)}{(\mathbf{V}\otimes \mathbf{U})}^{\top }\end{array}∥}_F^2+{\alpha }_W{∥\begin{array}{c}\mathbf{W}\end{array}∥}_F^2+{\lambda }_{W}Tr\left({\mathbf{W}}^{\top }{\mathbf{L}}_{\mathbf{W}}\mathbf{W}\right)+Tr\left({\lambda }_{\mathbf{W}}\right).\end{array}$$

(26)

Since $\widehat{\mathcal{Q}}\left({\mathbf{W}}^{\prime }\right)$ in Equation (25) is a convex function regarding ${\mathbf{W}}^{\prime }$, and satisfies the L-Lipschitz condition [28], there exists a constant $L_2⩾0$ such that

$$∥\begin{array}{c}\nabla \widehat{\mathcal{Q}}\left({\mathbf{W}}^{\prime }\right)-\nabla \widehat{\mathcal{Q}}\left(\mathbf{W}\right)\end{array}∥⩽L_2∥\begin{array}{c}{\mathbf{W}}^{\prime }-\mathbf{W}\end{array}∥,\forall ({\mathbf{W}}^{\prime },\mathbf{W}),$$

(27)

where $\nabla \widehat{\mathcal{Q}}\left({\mathbf{W}}^{\prime }\right)$ is the derivative of ${\mathbf{W}}^{\prime }$. $\widehat{\mathcal{Q}}\left({\mathbf{W}}^{\prime }\right)$ is to be by the approximated second-order Taylor expansion near $\mathbf{W}$ as

$$\begin{array}{cc}\hfill \tilde{\mathcal{Q}}(\mathbf{W})\approx & \widehat{\mathcal{Q}}({\mathbf{W}}^{\prime })+{\displaystyle \frac{L_2}{2}}{(\mathbf{W}-{\mathbf{W}}^{\prime })}^2+{\displaystyle \frac{\partial \widehat{\mathcal{Q}}({\mathbf{W}}^{\prime })}{\partial ({\mathbf{W}}^{\prime })}}(\mathbf{W}-{\mathbf{W}}^{\prime })\hfill \\ & ={\displaystyle \frac{L_2}{2}}{(\mathbf{W}-({\mathbf{W}}^{\prime }-{\displaystyle \frac{1}{L_2}}{\displaystyle \frac{\partial \widehat{\mathcal{Q}}({\mathbf{W}}^{\prime })}{\partial {\mathbf{W}}^{\prime }}}))}^2+\Theta ({\mathbf{W}}^{\prime }),\hfill \\ & \Theta ({\mathbf{W}}^{\prime })=\widehat{\mathcal{Q}}({\mathbf{W}}^{\prime })-{\displaystyle \frac{1}{2L_2}}{({\displaystyle \frac{\partial \widehat{\mathcal{Q}}({\mathbf{W}}^{\prime })}{\partial {\mathbf{W}}^{\prime }}})}^2.\hfill \end{array}$$

(28)

Let ${\mathbf{W}}^{\prime }={\mathbf{W}}^{\left(t\right)}$, then, the minimum value of Equation (28) is obtained as ${\mathbf{W}}^{(t+1)}$ and

$${\mathbf{W}}^{(t+1)}={\mathbf{W}}^{\left(t\right)}-{\displaystyle \frac{1}{L_2}}{\displaystyle \frac{\partial \widehat{\mathcal{Q}}\left({\mathbf{W}}^t\right)}{\partial \left({\mathbf{W}}^t\right)}}.$$

(29)

Similarly, since $\Psi \left({\mathbf{W}}^{\left(t\right)}\right)$ is independent of ${\mathbf{W}}^t$, the objective function updated with $\mathbf{W}$ can be written as

$$\begin{array}{c}\hfill {\mathbf{W}}^{(t+1)}=arg\underset{\mathbf{W}}{min}[{\displaystyle \frac{L_2}{2}}{∥\begin{array}{c}\mathbf{W}-({\mathbf{W}}^{\left(t\right)}-{\displaystyle \frac{1}{L_2}}{\displaystyle \frac{\partial \widehat{\mathcal{Q}}\left({\mathbf{W}}^{\left(t\right)}\right)}{\partial {\mathbf{W}}^{\left(t\right)}}})\end{array}∥}_2^2+{\gamma }_W{∥\begin{array}{c}\mathbf{W}\end{array}∥}_1^1].\end{array}$$

(30)

Let ${\mathbf{R}}^{\left(t\right)}={\mathbf{W}}^{\left(t\right)}-{\displaystyle \frac{1}{L_2}}{\displaystyle \frac{\partial \widehat{\mathcal{Q}}\left({\mathbf{W}}^t\right)}{\partial {\mathbf{W}}^{\left(t\right)}}}$, then, for solving Equation (30), we first solve ${\mathbf{R}}^{\left(t\right)}$ and then

$$\begin{array}{cc}\hfill {\mathbf{W}}^{(t+1)}=& arg\underset{\mathbf{W}}{min}[{\displaystyle \frac{L_2}{2}}{∥\begin{array}{c}\mathbf{W}-{\mathbf{R}}^{\left(t\right)}\end{array}∥}_2^2+{\gamma }_W{∥\begin{array}{c}\mathbf{W}\end{array}∥}_1^1].\hfill \end{array}$$

(31)

Now, the partial derivative of $\widehat{\mathcal{Q}}\left({\mathbf{W}}^{\left(t\right)}\right)$ to ${\mathbf{W}}^{\left(t\right)}$ is

$$\begin{array}{cc}\hfill {\displaystyle \frac{\partial \widehat{\mathcal{Q}}\left({\mathbf{W}}^{\left(t\right)}\right)}{\partial \left({\mathbf{W}}^{\left(t\right)}\right)}}=& -2({\mathbf{X}}_{\left(3\right)}^{\left(t\right)}-{\mathbf{W}}^{\left(t\right)}{\mathbf{G}}_{\left(3\right)}^{\left(t\right)}{({\mathbf{V}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)})}^{\top }){\left({\mathbf{G}}_{\left(3\right)}^{\left(t\right)}{({\mathbf{V}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)})}^{\top }\right)}^{\top }+\hfill \\ & 2{\alpha }_W{\mathbf{W}}^{\left(t\right)}+2{\lambda }_W{\mathbf{L}}_{{\mathbf{W}}^{\left(t\right)}}{\mathbf{W}}^{\left(t\right)}+{\lambda }_{{\mathbf{W}}^{\left(t\right)}}\hfill \\ & =-2{\mathbf{X}}_{\left(3\right)}^{\left(t\right)}({\mathbf{V}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)}){{\mathbf{G}}_{\left(3\right)}^{\left(t\right)}}^{\top }+\hfill \\ & 2{\mathbf{W}}^{\left(t\right)}{\mathbf{G}}_{\left(3\right)}^{\left(t\right)}{({\mathbf{V}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)})}^{\top }({\mathbf{V}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)}){{\mathbf{G}}_{\left(3\right)}^{\left(t\right)}}^{\top }+\hfill \\ & 2{\alpha }_3{\mathbf{W}}^{\left(t\right)}+2{\lambda }_W({\mathbf{D}}_{\mathbf{W}}^{\left(t\right)}-{\mathbf{C}}_{\mathbf{W}}^{\left(t\right)}){\mathbf{W}}^{\left(t\right)}+{\lambda }_{\mathbf{W}}^{\left(t\right)},\hfill \end{array}$$

(32)

where ${\mathbf{D}}_{\mathbf{W}}^{\left(t\right)}$ and ${\mathbf{C}}_{\mathbf{W}}^{\left(t\right)}$ should be solved at the same time. According to the (Karush–Kuhn–Tucker) (KKT) condition [27], ${\lambda }_{\mathbf{W}}{\mathbf{W}}^{\left(t\right)}=0$, and  ${\displaystyle \frac{\partial \widehat{\mathcal{Q}}\left({\mathbf{W}}^{\left(t\right)}\right)}{\partial {\mathbf{W}}^{\left(t\right)}}}=0$, we can obtain the following:

$$\begin{array}{cc}\hfill & (-2{\mathbf{X}}_{\left(3\right)}^{\left(t\right)}({\mathbf{V}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)}){{\mathbf{G}}_{\left(3\right)}^{\left(t\right)}}^{\top }-2{\lambda }_W{\mathbf{C}}_{\mathbf{W}}^{\left(t\right)}{\mathbf{W}}^{\left(t\right)}+\hfill \\ & 2{\mathbf{W}}^{\left(t\right)}{\mathbf{G}}_{\left(3\right)}^{\left(t\right)}{({\mathbf{V}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)})}^{\top }({\mathbf{V}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)}){{\mathbf{G}}_{\left(\mathbf{3}\right)}^{\left(\mathbf{t}\right)}}^{\top }+2{\alpha }_W{{\mathbf{A}}^{\left(3\right)}}^{\left(t\right)}+\hfill \\ & 2{\lambda }_W{\mathbf{D}}_{\mathbf{W}}^{\left(t\right)}{\mathbf{W}}^{\left(t\right)}){\mathbf{W}}^{\left(t\right)}=0.\hfill \end{array}$$

(33)

Therefore, the update rule of ${\mathbf{R}}^{\left(t\right)}$ is

$$\begin{array}{c}\hfill {\mathbf{R}}_{ij}^{\left(t\right)}={{\mathbf{W}}_{ij}}^{\left(t\right)}{\displaystyle \frac{{[{{\mathbf{X}}_{\left(3\right)}}^{\left(t\right)}({\mathbf{V}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)}){{\mathbf{G}}_{\left(3\right)}^{\left(t\right)}}^{\top }+{\lambda }_W{\mathbf{C}}_{\mathbf{W}}^{\left(t\right)}{\mathbf{W}}^{\left(t\right)}]}_{ij}}{{[{\mathbf{W}}^{\left(t\right)}{\mathbf{G}}_{\left(3\right)}^{\left(t\right)}{({\mathbf{V}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)})}^{\top }({\mathbf{V}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)}){{\mathbf{G}}_{\left(3\right)}^{\left(t\right)}}^{\top }+{\alpha }_W{\mathbf{W}}^{\left(t\right)}+{\lambda }_W{\mathbf{D}}_{\mathbf{W}}^{\left(t\right)}{\mathbf{W}}^{\left(t\right)}]}_{ij}}}.\end{array}$$

(34)

Similarly, we minimize ${\mathbf{R}}_{ij}^{\left(t\right)}$ and ${∥\begin{array}{c}\mathbf{W}\end{array}∥}_1^1$ using gradient descent and soft thresholding, respectively. This leads to the following update rule for $\mathbf{W}$:  

$${{\mathbf{W}}_{ij}}^{(t+1)}=\left\{\begin{array}{cc}{\mathbf{R}}_{ij}^{\left(t\right)}-{\displaystyle \frac{{\gamma }_W}{L_2}},\hfill & {\mathbf{R}}_{ij}^{\left(t\right)}>{\displaystyle \frac{{\gamma }_W}{L_2}},\hfill \\ 0,\hfill & {\mathbf{R}}_{ij}^{\left(t\right)}\le {\displaystyle \frac{{\gamma }_W}{L_2}}.\hfill \end{array}\right.$$

(35)

3.4. Algorithm Design

According to the above reasoning, a clustering algorithm based on GDSNTD is summarized, with its detailed steps outlined in Algorithm 1.

Algorithm 1: GDSNTD for Clustering

Input:       Data tensor $\mathcal{X}\in {\mathbb{R}}_{+}^{I_1\times I_2\times I_3}$. The algorithm parameters $J_1$, $J_2$, $J_3$, p, and regularization      parameters ${\alpha }_V$, ${\alpha }_W$, ${\lambda }_V$, ${\lambda }_W$, ${\gamma }_V$, ${\gamma }_W$. The stopping criterion and the maximum number       of iterations maxiter. Output: Core tensor $\mathcal{G}$ and factors $\mathbf{U},\mathbf{V},\mathbf{W}$; Ensure:      Clustering results;      repeat      for $k=1,2,\dots ,$ maxiter do 1:     Update $\mathcal{G},\mathbf{U},\mathbf{V},\mathbf{W}$, and using (8), (13), (24), (35), respectively.       if $re<ϵ$            Return $\mathcal{G},\mathbf{U},\mathbf{V},\mathbf{W}$       end if      end for      until converges 2:     Run k-means on $\mathbf{W}$ to get clustering result. 3:     return Clustering results

We employ a stopping criterion based on the relative change in the objective function value. The criterion is defined as

$$re={\displaystyle \frac{f^{(k-1)}-f^{\left(k\right)}}{f^{\left(k\right)}}},$$

where $f^{(k-1)}$ and $f^{\left(k\right)}$ are the objective function values at the $(k-1)$-th and k-th iterations, respectively. The iterations are terminated when $re<ϵ$, with $ϵ$ being a predefined convergence tolerance [29].

3.5. Convergence Analysis

Since the update rule for $\mathcal{G}$ is identical to that in [17], this subsection employs the auxiliary function approach exclusively to prove the convergence of the updating rules given in Equations (8), (13), (24) and (35). We now present the following theorem:

Theorem 1.

For $\mathcal{G}⩾0$, $\mathbf{U}⩾0$, $\mathbf{V}⩾0$, and $\mathbf{W}⩾0$, the objective function in (2) is non-increasing under the updating rules in (8), (13), (24), and (35) if and only if $\mathbf{U}⩾0$, $\mathbf{V}⩾0$, $\mathbf{W}⩾0$, and $\mathcal{G}$ are at a stationary point.

To prove Theorem 1, we first give a definition and several lemmas.

Definition 1

([1]). $\mathcal{H}(a,a^{\prime })$ is an auxiliary function for $\mathcal{J}\left(a\right)$ if the conditions $\mathcal{H}(a,a^{\prime })\ge \mathcal{J}\left(a\right)$ and $\mathcal{H}(a,a)=\mathcal{J}\left(a\right)$ are satisfied.

Lemma 1

([1]). If $\mathcal{H}(a,a^{\prime })$ is an auxiliary function for $\mathcal{J}\left(a\right)$, then $\mathcal{J}\left(a\right)$ is non-increasing under the updating rule

$$a^{t+1}=arg\underset{a}{min}\mathcal{H}(a,a^t).$$

(36)

Proof. 

$\mathcal{J}\left(a^{t+1}\right)\le \mathcal{H}(a^{t+1},a^t)\le \mathcal{H}(a^t,a^t)\le \mathcal{J}\left(a^t\right)$. □

The equality $\mathcal{J}\left(a^{t+1}\right)=\mathcal{J}\left(a^t\right)$ holds only if $a^t$ is a local minimum of $\mathcal{H}(a,a^t)$. By iterating the update rule (36), $a^t$ converges to the local minimum of $\mathcal{H}(a,a^t)$.

Next, we provide a detailed proof of the convergence for the above updating rule for $\mathbf{U}$ using an auxiliary function. Recall the function from Equation (3):

$$\widehat{\mathcal{J}}\left(\mathbf{U}\right)={∥{\mathbf{X}}_{\left(1\right)}-\mathbf{U}{\mathbf{G}}_{\left(1\right)}{\left(\mathbf{W}\otimes \mathbf{V}\right)}^{\top }∥}_F^2.$$

(37)

We can obtain

$$\begin{array}{cc}\hfill {\widehat{\mathcal{J}}}_{ij}^{\prime }\left({{\mathbf{U}}_{ij}}^{\left(t\right)}\right)& ={\displaystyle \frac{\partial \widehat{\mathcal{J}}\left({{\mathbf{U}}_{ij}}^{\left(t\right)}\right)}{\partial \left({{\mathbf{U}}_{ij}}^{\left(t\right)}\right)}}=[-2{\mathbf{X}}_{\left(1\right)}^{\left(t\right)}{\left(\mathbf{W}\otimes \mathbf{V}\right)}^{\left(t\right)}{{{\mathbf{G}}_{\left(1\right)}}^{\left(t\right)}}^{\top }+\hfill \\ & 2{\mathbf{U}}^{\left(t\right)}{{\mathbf{G}}_{\left(1\right)}}^{\left(t\right)}\left({{\left(\mathbf{W}\otimes \mathbf{V}\right)}^{\left(t\right)}}^{\top }{\left(\mathbf{W}\otimes \mathbf{V}\right)}^{\left(t\right)}\right){{{\mathbf{G}}_{\left(r\right)}}^{\left(t\right)}}^{\top }{]}_{ij}.\hfill \end{array}$$

(38)

$$\begin{array}{cc}\hfill {\widehat{\mathcal{J}}}_{ij}^{\prime \prime }\left({{\mathbf{U}}_{ij}}^{\left(t\right)}\right)=& {\left[2{{\mathbf{G}}_{\left(1\right)}}^{\left(t\right)}\left({{\left(\mathbf{W}\otimes \mathbf{V}\right)}^{\left(t\right)}}^{\top }{\left(\mathbf{W}\otimes \mathbf{V}\right)}^{\left(t\right)}\right){{{\mathbf{G}}_{\left(1\right)}}^{\left(t\right)}}^{\top }\right]}_{jj}.\hfill \end{array}$$

(39)

Essentially, it is sufficient to prove that each ${\widehat{\mathcal{J}}}_{ij}\left({\mathbf{U}}_{ij}\right)$ is non-increasing under the update rule.

Lemma 2.

The function

$$\begin{array}{cc}\hfill \widehat{\mathcal{H}}({\mathbf{U}}_{ij},{{\mathbf{U}}_{ij}}^t)=& {\widehat{\mathcal{J}}}_{ij}\left({{\mathbf{U}}_{ij}}^t\right)+{\widehat{\mathcal{J}}}_{ij}^{\prime }\left({{\mathbf{U}}_{ij}}^t\right)({\mathbf{U}}_{ij}-{{\mathbf{U}}_{ij}}^t)+\hfill \\ & {\displaystyle \frac{{\left[{\mathbf{U}}^t{\mathbf{G}}_{\left(1\right)}\left({(\mathbf{W}\otimes \mathbf{V})}^{\top }(\mathbf{W}\otimes \mathbf{V})\right){{\mathbf{G}}_{\left(1\right)}}^{\top }\right]}_{jj}}{{{\mathbf{U}}_{ij}}^t}}{({\mathbf{U}}_{ij}-{{\mathbf{U}}_{ij}}^t)}^2\hfill \end{array}$$

(40)

is an auxiliary function for ${\widehat{\mathcal{J}}}_{ij}\left({\mathbf{U}}_{ij}\right)$, where the matrix ${\mathbf{U}}^t=\left({{\mathbf{U}}_{ij}}^t\right)$.

Proof. 

Obviously, $\tilde{\mathcal{H}}({{\mathbf{U}}_{ij}}^t,{{\mathbf{U}}_{ij}}^t)={\widehat{\mathcal{J}}}_{ij}\left({{\mathbf{U}}_{ij}}^t\right)$. According to Definition 1, we only need to show that $\tilde{\mathcal{H}}({\mathbf{U}}_{ij},{{\mathbf{U}}_{ij}}^t)$≥${\widehat{\mathcal{J}}}_{ij}\left({\mathbf{U}}_{ij}\right)$. We first obtain the Taylor series expansion of ${\widehat{\mathcal{J}}}_{ij}\left({\mathbf{U}}_{ij}\right)$ at ${{\mathbf{U}}_{ij}}^t$ to be

$$\begin{array}{cc}\hfill & {\tilde{\mathcal{J}}}_{ij}\left({\mathbf{U}}_{ij}\right)={\mathcal{J}}_{ij}\left({{\mathbf{U}}_{ij}}^t\right)+{\widehat{\mathcal{J}}}_{ij}^{\prime }\left({{\mathbf{U}}_{ij}}^t\right)({\mathbf{U}}_{ij}-{{\mathbf{U}}_{ij}}^t)+{\displaystyle \frac{1}{2}}{\mathcal{J}}_{ij}^{\prime \prime }\left({{\mathbf{U}}_{ij}}^t\right){({\mathbf{U}}_{ij}-{{\mathbf{U}}_{ij}}^t)}^2\hfill \\ & ={\mathcal{J}}_{ij}\left({{\mathbf{U}}_{ij}}^t\right)+{\mathcal{J}}_{ij}^{\prime }\left({{\mathbf{U}}_{ij}}^t\right)({\mathbf{U}}_{ij}-{{\mathbf{U}}_{ij}}^t)+{\left[{\mathbf{G}}_{\left(1\right)}\left({(\mathbf{W}\otimes \mathbf{V})}^{\top }(\mathbf{W}\otimes \mathbf{V})\right){{\mathbf{G}}_{\left(1\right)}}^{\top }\right]}_{jj}{({\mathbf{U}}_{ij}-{{\mathbf{U}}_{ij}}^t)}^2.\hfill \end{array}$$

(41)

By Equation (40), $\mathcal{H}({\mathbf{U}}_{ij},{{\mathbf{U}}_{ij}}^t)$≥${\mathcal{J}}_{ij}\left({\mathbf{U}}_{ij}\right)$ is equivalent to

$$\begin{array}{c}\hfill {\displaystyle \frac{{\left[{\mathbf{U}}^t{\mathbf{G}}_{\left(1\right)}\left({(\mathbf{W}\otimes \mathbf{V})}^{\top }(\mathbf{W}\otimes \mathbf{V})\right){\mathbf{G}}_{\left(1\right)}^{\top }\right]}_{jj}}{{{\mathbf{U}}_{ij}}^t}}\ge {\left[{\mathbf{G}}_{\left(r\right)}\left({(\mathbf{W}\otimes \mathbf{V})}^{\top }(\mathbf{W}\otimes \mathbf{V})\right){\mathbf{G}}_{\left(1\right)}^{\top }\right]}_{jj}.\end{array}$$

(42)

We can obtain

$$\begin{array}{cc}\hfill {\left[{\mathbf{U}}^t{\mathbf{G}}_{\left(1\right)}\left({(\mathbf{W}\otimes \mathbf{V})}^{\top }(\mathbf{W}\otimes \mathbf{V})\right){\mathbf{G}}_{\left(1\right)}^{\top }\right]}_{jj}& ={{\mathbf{U}}_{il}}^t{\left[{\mathbf{G}}_{\left(1\right)}\left({(\mathbf{W}\otimes \mathbf{V})}^{\top }(\mathbf{W}\otimes \mathbf{V})\right){\mathbf{G}}_{\left(1\right)}^{\top }\right]}_{lj}\hfill \\ & \ge {{\mathbf{U}}_{ij}}^t{\left[{\mathbf{G}}_{\left(1\right)}\left({(\mathbf{W}\otimes \mathbf{V})}^{\top }(\mathbf{W}\otimes \mathbf{V})\right){\mathbf{G}}_{\left(1\right)}^{\top }\right]}_{jj},\hfill \end{array}$$

(43)

Therefore, the inequality $\tilde{\mathcal{H}}({\mathbf{U}}_{ij},{{\mathbf{U}}_{ij}}^t)$≥${\widehat{\mathcal{J}}}_{ij}\left({\mathbf{U}}_{ij}\right)$ holds. □

Analogously, the convergence under the update rule in Equation (24) can be proven. Next, we provide a detailed proof of convergence for the $\mathbf{V}$ update rule using an auxiliary function. Consider the following function:

$$\mathcal{J}\left(\mathcal{V}\right)=\widehat{\mathcal{J}}+{\gamma }_V{∥\begin{array}{c}\mathbf{V}\end{array}∥}_1^1,$$

(44)

where

$$\begin{array}{c}\hfill \widehat{\mathcal{J}}\left(\mathbf{V}\right)={∥\begin{array}{c}{\mathbf{X}}_{\left(2\right)}-\mathbf{V}{\mathbf{G}}_{\left(2\right)}{(\mathbf{W}\otimes \mathbf{U})}^{\top }\end{array}∥}_F^2+{\alpha }_V{∥\begin{array}{c}\mathbf{U}\end{array}∥}_F^2+{\lambda }_{V}Tr\left({\mathbf{V}}^{\top }{\mathbf{L}}_{\mathbf{V}}\mathbf{V}\right),\end{array}$$

(45)

We derive

$$\begin{array}{cc}\hfill {\widehat{\mathcal{J}}}_{ij}^{\prime }\left({{\mathbf{V}}_{ij}}^{\left(t\right)}\right)=& {\displaystyle \frac{\partial {\widehat{\mathcal{J}}}_{ij}\left({{\mathbf{V}}_{ij}}^{\left(t\right)}\right)}{\partial \left({{\mathbf{V}}_{ij}}^{\left(t\right)}\right)}}=[-2{\mathbf{X}}_{\left(2\right)}^{\left(t\right)}({\mathbf{W}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)}){{\mathbf{G}}_{\left(2\right)}^{\left(t\right)}}^{\top }+\hfill \\ & 2{\mathbf{V}}^{\left(t\right)}{\mathbf{G}}_{\left(2\right)}^{\left(t\right)}\left({({\mathbf{W}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)})}^{\top }({\mathbf{W}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)})\right){{\mathbf{G}}_{\left(2\right)}^{\left(t\right)}}^{\top }+\hfill \\ & 2{\alpha }_V{\mathbf{V}}^{\left(t\right)}+2{\lambda }_V{\mathbf{D}}_{\mathbf{V}}^{\left(t\right)}{\mathbf{V}}^{\left(t\right)}-2{\lambda }_V{\mathbf{C}}_{\mathbf{V}}^{\left(t\right)}{\mathbf{V}}^{\left(t\right)}{]}_{ij},\hfill \end{array}$$

(46)

$$\begin{array}{cc}\hfill {\widehat{\mathcal{J}}}_{ij}^{\prime \prime }\left({{\mathbf{V}}_{ij}}^{\left(t\right)}\right)=& [2{\mathbf{G}}_{\left(2\right)}^{\left(t\right)}{({\mathbf{W}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)})}^T({\mathbf{W}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)}){{\mathbf{G}}_{\left(2\right)}^{\left(t\right)}}^T+\hfill \\ & 2{\alpha }_V+2{\lambda }_V{\mathbf{D}}_{\mathbf{V}}^{\left(t\right)}-2{\lambda }_V{\mathbf{C}}_{\mathbf{V}}^{\left(t\right)}{]}_{jj}.\hfill \end{array}$$

(47)

Essentially, the updating rule is element wise, so it is sufficient to prove each ${\widehat{\mathcal{J}}}_{ij}\left({\mathbf{V}}_{ij}\right)$ is non-increasing under the update rule.

Lemma 3.

The function

$$\begin{array}{c}\hfill \widehat{\mathcal{H}}({\mathbf{V}}_{ij},{{\mathbf{V}}_{ij}}^t)={\widehat{\mathcal{J}}}_{ij}\left({{\mathbf{V}}_{ij}}^t\right)+{\widehat{\mathcal{J}}}_{ij}^{\prime }\left({{\mathbf{V}}_{ij}}^t\right)({\mathbf{V}}_{ij}-{{\mathbf{V}}_{ij}}^t)+{\displaystyle \frac{M_1+M_2}{{{\mathbf{V}}_{ij}}^t}}{({\mathbf{V}}_{ij}-{{\mathbf{V}}_{ij}}^t)}^2\end{array}$$

(48)

is an auxiliary function for ${\widehat{\mathcal{J}}}_{ij}\left({\mathbf{V}}_{ij}\right)$, where the matrix ${\mathbf{V}}^t=\left({{\mathbf{V}}_{ij}}^t\right)$, and

$$M_1={\mathbf{V}}^{\left(t\right)}{\mathbf{G}}_{\left(2\right)}^{\left(t\right)}{({\mathbf{W}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)})}^{\top }({\mathbf{W}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)}){{\mathbf{G}}_{\left(2\right)}^{\left(t\right)}}^{\top },$$

$$M_2={\alpha }_V{\mathbf{V}}^{\left(t\right)}+{\lambda }_V{\mathbf{D}}_{\mathbf{V}}^{\left(t\right)}{\mathbf{V}}^{\left(t\right)}.$$

Proof. 

Obviously, $\tilde{\mathcal{H}}({{\mathbf{V}}_{ij}}^t,{{\mathbf{V}}_{ij}}^t)={\widehat{\mathcal{J}}}_{ij}\left({{\mathbf{V}}_{ij}}^t\right)$. According to Definition 1, we only need to show that $\tilde{\mathcal{H}}({\mathbf{V}}_{ij},{{\mathbf{V}}_{ij}}^t)$≥${\widehat{\mathcal{J}}}_{ij}\left({\mathbf{V}}_{ij}\right)$. We first obtain the Taylor series expansion of ${\widehat{\mathcal{J}}}_{ij}\left({\mathbf{V}}_{ij}\right)$ at ${{\mathbf{V}}_{ij}}^t$ to be

$$\begin{array}{cc}\hfill & {\tilde{\mathcal{J}}}_{ij}\left({\mathbf{V}}_{ij}\right)={\mathcal{J}}_{ij}\left({{\mathbf{V}}_{ij}}^t\right)+{\widehat{\mathcal{J}}}_{ij}^{\prime }\left({{\mathbf{V}}_{ij}}^t\right)({\mathbf{V}}_{ij}-{{\mathbf{V}}_{ij}}^t)+{\displaystyle \frac{1}{2}}{\mathcal{J}}_{ij}^{\prime \prime }\left({{\mathbf{V}}_{ij}}^t\right){({\mathbf{V}}_{ij}-{{\mathbf{V}}_{ij}}^t)}^2\hfill \\ & ={\mathcal{J}}_{ij}\left({{\mathbf{V}}_{ij}}^t\right)+{\mathcal{J}}_{ij}^{\prime }\left({{\mathbf{V}}_{ij}}^t\right)({\mathbf{V}}_{ij}-{{\mathbf{V}}_{ij}}^t)+[{\mathbf{G}}_{\left(2\right)}^{\left(t\right)}\left({({\mathbf{W}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)})}^{\top }({\mathbf{W}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)})\right){{\mathbf{G}}_{\left(2\right)}^{\left(t\right)}}^{\top }\hfill \\ & +{\alpha }_V+{\lambda }_V{\mathbf{D}}_{\mathbf{V}}^{\left(t\right)}-{\lambda }_V{\mathbf{C}}_{\mathbf{V}}^{\left(t\right)}{]}_{jj}{({\mathbf{V}}_{ij}-{{\mathbf{V}}_{ij}}^t)}^2.\hfill \end{array}$$

(49)

By Equation (48) and Equation (49), $\mathcal{H}({\mathbf{V}}_{ij},{{\mathbf{V}}_{ij}}^t)$≥${\mathcal{J}}_{ij}\left({\mathbf{V}}_{ij}\right)$ is equivalent to

$$\begin{array}{c}\hfill {\displaystyle \frac{{[M_1+M_2]}_{ij}}{{{\mathbf{V}}_{ij}}^t}}\ge {[{\mathbf{G}}_{\left(2\right)}^{\left(t\right)}{({\mathbf{W}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)})}^{\top }({\mathbf{W}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)}){{\mathbf{G}}_{\left(2\right)}^{\left(t\right)}}^{\top }+{\alpha }_V+{\lambda }_V{\mathbf{D}}_{\mathbf{V}}^{\left(t\right)}-{\lambda }_V{\mathbf{C}}_{\mathbf{V}}^{\left(t\right)}]}_{jj}.\end{array}$$

(50)

We can obtain

$$\begin{array}{cc}\hfill & {\left[{\mathbf{V}}^{\left(t\right)}{\mathbf{G}}_{\left(2\right)}^{\left(t\right)}\left({({\mathbf{W}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)})}^{\top }({\mathbf{W}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)})\right){{\mathbf{G}}_{\left(2\right)}^{\left(t\right)}}^{\top }\right]}_{ij}\hfill \\ & ={{\mathbf{V}}_{il}}^t{\left[{\mathbf{G}}_{\left(2\right)}^{\left(t\right)}\left({({\mathbf{W}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)})}^{\top }({\mathbf{W}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)})\right){{\mathbf{G}}_{\left(2\right)}^{\left(t\right)}}^{\top }\right]}_{lj}\hfill \\ & \ge {{\mathbf{V}}_{ij}}^t{\left[{\mathbf{G}}_{\left(2\right)}^{\left(t\right)}\left({({\mathbf{W}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)})}^{\top }({\mathbf{W}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)})\right){{\mathbf{G}}_{\left(2\right)}^{\left(t\right)}}^{\top }\right]}_{jj},\hfill \end{array}$$

(51)

and

$$\begin{array}{c}\hfill {\left[{\lambda }_V{\mathbf{D}}_{\mathbf{V}}^{\left(t\right)}{\mathbf{V}}^{\left(t\right)}\right]}_{ij}={{\mathbf{V}}_{il}}^t{\left[{\lambda }_V{\mathbf{D}}_{\mathbf{V}}^{\left(t\right)}\right]}_{lj}\ge {{\mathbf{V}}_{ij}}^t{\left[{\lambda }_V{\mathbf{D}}_{\mathbf{V}}^{\left(t\right)}\right]}_{jj}\ge {{\mathbf{V}}_{ij}}^t{\left[{\lambda }_V({\mathbf{D}}_{\mathbf{V}}^{\left(t\right)}-{\mathbf{V}}_{\mathbf{V}}^{\left(t\right)})\right]}_{jj},\end{array}$$

(52)

Therefore, the inequality $G({\mathbf{V}}_{ij},{{\mathbf{V}}_{ij}}^t)$≥$F_{ij}\left({\mathbf{V}}_{ij}\right)$ holds. □

Thus, Equation (50) holds, and $\mathcal{H}({\mathbf{V}}_{ij},{{\mathbf{V}}_{ij}}^t)$≥${\mathcal{J}}_{ij}\left({\mathbf{V}}_{ij}\right)$. According to Equation (23),

$$\begin{array}{cc}\hfill {\mathbf{S}}_{ij}^{\left(t\right)}=& {{\mathbf{V}}_{ij}}^{\left(t\right)}{\displaystyle \frac{{[{{\mathbf{X}}_{\left(2\right)}}^{\left(t\right)}({\mathbf{W}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)}){{\mathbf{G}}_{\left(2\right)}^{\left(t\right)}}^{\top }+{\lambda }_V{\mathbf{C}}_{\mathbf{V}}^{\left(t\right)}{\mathbf{V}}^{\left(t\right)}]}_{ij}}{{[{\mathbf{V}}^{\left(t\right)}{\mathbf{G}}_{\left(2\right)}^{\left(t\right)}{({\mathbf{W}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)})}^{\top }({\mathbf{W}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)}){{\mathbf{G}}_{\left(2\right)}^{\left(t\right)}}^{\top }+{\alpha }_V{\mathbf{V}}^{\left(t\right)}+{\lambda }_V{\mathbf{D}}_{\mathbf{V}}^{\left(t\right)}{\mathbf{V}}^{\left(t\right)}]}_{ij}}}\hfill \\ & ={{\mathbf{V}}_{ij}}^{\left(t\right)}-{{\mathbf{V}}_{ij}}^{\left(t\right)}{\displaystyle \frac{{\widehat{\mathcal{J}}}_{ij}^{\prime }\left({{\mathbf{V}}_{ij}}^{\left(t\right)}\right)}{{[{\mathbf{V}}^{\left(t\right)}{\mathbf{G}}_{\left(2\right)}^{\left(t\right)}{({\mathbf{W}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)})}^{\top }({\mathbf{W}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)}){{\mathbf{G}}_{\left(2\right)}^{\left(t\right)}}^{\top }+{\alpha }_V{\mathbf{V}}^{\left(t\right)}+{\lambda }_V{\mathbf{D}}_{\mathbf{V}}^{\left(t\right)}{\mathbf{V}}^{\left(t\right)}]}_{ij}}}.\hfill \end{array}$$

(53)

The soft thresholding operation ensures convergence as follows: if ${\mathbf{S}}_{ij}^{\left(t\right)}>{\displaystyle \frac{{\gamma }_V}{L}}$, the update subtracts a constant value; otherwise, if ${\mathbf{S}}_{ij}^{\left(t\right)}\le {\displaystyle \frac{{\gamma }_V}{L}}$, the value is set to zero. Both cases are consistent with the convergence proof. Because Equation (51) is an auxiliary function for ${\widehat{\mathcal{J}}}_{ij}$, and according to the soft threshold, ${\mathcal{J}}_{ij}$ is non-increasing under the updating rule stated in Equation (24). Analogously, we can prove the convergence under the updating rule in Equation (35).

4. Experiments

In this section, the experimental setup and analysis of the results are discussed in detail. The descriptions of these datasets are given as follows. First, we introduce the image datasets Coil20 (http://www.cad.zju.edu.cn/home/dengcai/Data/MLData.html, (accessed on 15 May 2024)), Georgia (https://www.cnblogs.com/kuangqiu/p/7776829.html, (accessed on 10 March 2025)), Iris (http://archive.ics.uci.edu/dataset/53/iris, (accessed 20 November 2024)) [30]. The datasets are summarized in

Table 1. Summary of the datasets.

Idx	Datasets	Samples	Features	Classes
D01	Georgia	750	1024	50
D02	Coil20	1440	1024	20
D03	Iris	150	4	3

.

The details of the public datasets used in the experiments are presented in Figure 1 and Figure 2.

In order to evaluate the effectiveness of our proposed GDSNTD scheme, we compared it with six classical or state-of-the-art clustering and co-clustering methods. All the simulations were performed on a computer with a 2.30-GHz Intel Core i7-11800H CPU and 32 GB memory of 64-bit MATLAB 2016a in Windows 10. Without special specifications, the maximum number of iterations is set to 1000.

• Non-negative matrix factorization (NMF) [1]: NMF aims to decompose a matrix into two low-dimensional matrices and is now often used as a data processing method in machine learning.

  $$F_{NMF}=\underset{\mathbf{U}⩾0,\mathbf{V}⩾0}{min}∥\begin{array}{c}\mathbf{X}-\mathbf{U}{\mathbf{V}}^{\top }\end{array}∥.$$
• Non-negative Tucker decomposition (NTD) [7]: The NTD algorithm is considered as a generalization of NMF.

  $$F_{NTD}=\underset{\mathcal{G}⩾0,\mathbf{U},\mathbf{V},\mathbf{W}⩾0}{min}{∥\begin{array}{c}\mathcal{X}-\mathcal{G}{\times }_1\mathbf{U}{\times }_2\mathbf{V}{\times }_3\mathbf{W}\end{array}∥}_F^2.$$
• Graph-regularized NTD (GNTD) [17]:

  $$F_{GNTD}=\underset{\mathcal{G}⩾0,\mathbf{U},\mathbf{V},\mathbf{W}⩾0}{min}{∥\begin{array}{c}\mathcal{X}-\mathcal{G}{\times }_1\mathbf{U}{\times }_2\mathbf{V}{\times }_3\mathbf{W}\end{array}∥}_F^2+\lambda Tr\left({\mathbf{W}}^{\top }L\mathbf{W}\right).$$
• Graph dual regularized NMF (GDNMF) [12]: GDNMF simultaneously considers the geometric structures of both the data manifold and the feature manifold.

  $$F_{GDNMF}=\underset{\mathbf{U}⩾0,\mathbf{V}⩾0}{min}{∥\begin{array}{c}\mathbf{X}-\mathbf{U}{\mathbf{V}}^{\top }\end{array}∥}_F^2+{\lambda }_{1}Tr\left({\mathbf{V}}^{\top }{\mathbf{L}}_{\mathbf{V}}\mathbf{V}\right)+{\lambda }_{2}Tr\left({\mathbf{U}}^{\top }{\mathbf{L}}_{\mathbf{U}}\mathbf{U}\right).$$
• Graph dual regularized non-negative matrix tri-factorization (GDNMTF) [12]: DNMTF is an extension of our DNMF algorithm and simultaneously incorporates two graph regularizers of both data manifold and feature manifold into its objective function.

  $$F_{GDNMTF}=\underset{\mathbf{U}⩾0,\mathbf{S}⩾0,\mathbf{V}⩾0}{min}{∥\begin{array}{c}\mathbf{X}-\mathbf{U}\mathbf{S}{\mathbf{V}}^{\top }\end{array}∥}_F^2+{\lambda }_{1}Tr\left({\mathbf{V}}^{\top }{\mathbf{L}}_{\mathbf{V}}\mathbf{V}\right)+{\lambda }_{2}Tr\left({\mathbf{U}}^{\top }{\mathbf{L}}_{\mathbf{U}}\mathbf{U}\right).$$
• Graph-regularized sparse non-negative matrix trifactorization (GSNMTF) [21]: The GSNMTF model introduces graph regularization and an $l_1$ norm constraint into the objective function.

  $$F_{GSNMTF}=\underset{\mathbf{U}⩾0,\mathbf{S}⩾0,\mathbf{V}⩾0}{min}{∥\begin{array}{c}\mathbf{X}-\mathbf{U}\mathbf{S}{\mathbf{V}}^{\top }\end{array}∥}_F^2+{\lambda }_{U}Tr\left({\mathbf{U}}^{\top }{\mathbf{L}}_{\mathbf{U}}\mathbf{U}\right)+{\lambda }_{V}Tr\left({\mathbf{V}}^{\top }{\mathbf{L}}_{\mathbf{V}}\mathbf{V}\right)+$$

  $${\gamma }_U{∥\begin{array}{c}\mathbf{U}\end{array}∥}_1^1+{\gamma }_V{∥\begin{array}{c}\mathbf{V}\end{array}∥}_1^1.$$

4.1. Evaluation Measures

In this section, two widely used metrics, accuracy (AC) and Normalized Mutual Information (NMI), are used to evaluate the clustering performance. Accuracy aims to find a one-to-one relationship between classes and clusters. It is usually used to calculate the proportion of correct samples to the total number of samples. It is a relatively intuitive evaluation index. Accuracy is a clustering evaluation metric that measures the proportion of correctly assigned samples against the total, after a one-to-one mapping between clusters and true classes is established. It is an intuitive and widely used measure [31]. NMI is commonly used in clustering to measure the similarity between two clusterings.

AC is defined as

$$AC={\displaystyle \frac{1}{N}}\sum _{i=1}^N\delta (g{t}_i,map\left(c_i\right)),$$

where N is the total number of samples, $g{t}_i$ is the ground-truth label of sample i, $c_i$ is the cluster label assigned by the algorithm, $\delta (·,·)$ is the Dirac delta function, and $map(·)$ is the optimal mapping function [32].

NMI is defined as

$$NMI(B,T)={\displaystyle \frac{2·\left(I\right(B;T\left)\right)}{H\left(B\right)+H\left(T\right)}},$$

where $I(B;T)$ is the mutual information between B and T, and $H\left(B\right)$ and $H\left(T\right)$ are their entropies. A higher NMI indicates a better alignment between the clustering result and the true labels [33].

4.2. Experimental Setup and Clustering Results Analysis

In this part, the experimental setup is described, and the experimental results are discussed in detail. To ensure the fairness between models, our proposed algorithm and all comparison algorithms use the same random initialization matrix. Subsequently, each experiment is conducted ten times independently on original data, then K-means clustering is performed five times independently on this low-dimensional reduced data. The average and variance of the results are recorded. The standard deviation is set to 0 if it is less than $1\times {10}^{-5}$.

Table 2. AC (%) of each algorithm on each dataset.

Data	Classes	NMF	NTD	GNTD	GDNMF	GDNMTF	GSNMTF	GDSNTD
D01	50	37.18 ± 2.17	43.91 ± 2.10	52.66 ± 2.98	51.74 ± 2.39	53.39 ± 1.86	53.85 ± 1.79	55.37 ± 2.39
D02	20	61.19 ± 4.03	57.62 ± 4.15	78.06 ± 3.63	79.21 ± 3.62	78.95 ± 2.59	79.40 ± 3.05	79.85 ± 2.79
D03	3	82.13 ± 9.75	72.51 ± 8.81	93.61 ± 8.90	94.29 ± 1.10	96.11 ± 6.38	96.15 ± 3.05	96.27 ± 9.01

and

Table 3. NMI (%) of each algorithm on each dataset.

Data	Classes	NMF	NTD	GNTD	GDNMF	GDNMTF	GSNMTF	GDSNTD
D01	50	57.63 ± 1.35	63.47 ± 1.37	70.79 ± 1.95	69.66 ± 1.17	69.83 ± 1.09	70.02 ± 0.93	73.00 ± 1.15
D01	20	72.99 ± 1.80	71.32 ± 2.22	85.97 ± 1.70	88.07 ± 1.46	88.15 ± 1.06	88.05 ± 1.14	88.33 ± 1.07
D03	3	67.53 ± 5.78	60.69 ± 3.10	84.80 ± 6.12	87.40 ± 7.79	88.56 ± 4.59	88.37 ± 5.61	91.13 ± 7.14

summarize the accuracy and NMI results for each algorithm across all datasets.

Table 4. AC (%) of each algorithm on Georgia dataset.

Classes	NMF	NTD	GNTD	GDNMF	GDNMTF	GSNMTF	GDSNTD
3	80.44 ± 11.30	67.56 ± 11.84	87.24 ± 14.38	79.20 ± 12.61	81.29 ± 11.86	79.51 ± 12.52	91.82 ± 9.18
7	67.05 ± 8.76	58.51 ± 10.03	66.36 ± 11.23	64.63 ± 7.97	67.75 ± 9.12	69.22 ± 8.00	67.03 ± 12.54
11	55.5 ± 7.01	61.73 ± 7.87	70.29 ± 6.97	60.57 ± 6.70	61.26 ± 6.01	66.41 ± 6.58	70.64 ± 8.10
15	54.35 ± 7.02	53.5 ± 3.52	63.31 ± 5.65	59.37 ± 6.36	59.50 ± 6.61	58.45 ± 5.04	64.99 ± 5.49
19	49.36 ± 3.82	52.32 ± 6.86	60.45 ± 5.13	59.69 ± 5.30	58.81 ± 5.26	60.64 ± 4.40	61.95 ± 4.77
23	46.94 ± 4.51	49.01 ± 4.24	60.99 ± 4.70	55.91 ± 5.88	54.11 ± 3.79	55.43 ± 4.15	59.16 ± 5.65
27	44.78 ± 3.76	49.64 ± 2.52	58.21 ± 4.38	53.29 ± 4.02	57.12 ± 3.34	59.71 ± 3.01	60.25 ± 4.51
31	42.69 ± 3.85	46.65 ± 3.15	57.18 ± 4.11	55.31 ± 3.93	56.64 ± 3.10	56.68 ± 3.00	58.16 ± 3.82
35	41.64 ± 2.70	47.14 ± 3.60	55.05 ± 3.72	51.58 ± 4.71	56.45 ± 3.71	56.94 ± 3.70	57.31 ± 3.07
39	41.6 ± 2.68	44.74 ± 2.60	53.66 ± 3.22	50.90 ± 2.99	54.81 ± 2.98	54.57 ± 3.29	55.59 ± 3.07
43	39.88 ± 2.25	43.62 ± 3.17	53.35 ± 3.02	51.50 ± 2.70	54.13 ± 2.40	54.19 ± 2.42	54.20 ± 2.66
47	38.97 ± 2.42	43.81 ± 2.48	53.56 ± 3.22	50.51 ± 2.38	52.89 ± 1.89	53.17 ± 2.18	54.11 ± 2.30
49	37.53 ± 1.80	43.46 ± 2.22	52.60 ± 2.26	50.10 ± 2.85	53.39 ± 2.34	53.28 ± 2.43	52.88 ± 2.85
50	37.18 ± 2.17	43.91 ± 2.10	52.66 ± 2.98	51.74 ± 2.39	53.39 ± 1.86	53.85 ± 1.79	55.37 ± 2.39
Avg.	48.42	50.32	60.35	56.74	58.68	59.43	61.68

lists the accuracy and standard deviation for each algorithm on the Georgia dataset, while

Table 5. NMI (%) of each algorithm on Georgia dataset.

Classes	NMF	NTD	GNTD	GDNMF	GDNMTF	GSNMTF	GDSNTD
3	57.07 ± 20.27	49.33 ± 16.31	78.08 ± 21.42	65.85 ± 18.07	65.13 ± 15.61	65.00 ± 12.49	82.78 ± 15.77
7	60.4 ± 8.54	54.68 ± 10.73	65.48 ± 11.17	63.46 ± 7.16	66.87 ± 10.51	67.55 ± 10.18	67.1 ± 12.48
11	60.44 ± 8.04	65.62 ± 6.68	75.79 ± 5.91	65.82 ± 4.88	66.83 ± 4.85	70.14 ± 5.24	75.93 ± 6.89
15	60.79 ± 6.47	59.30 ± 2.84	71.45 ± 4.39	68.13 ± 5.05	67.71 ± 5.23	67.87 ± 4.26	73.05 ± 4.64
19	59.70 ± 2.31	62.60 ± 5.68	69.98 ± 4.13	70.93 ± 4.49	68.83 ± 3.81	71.29 ± 2.97	71.64 ± 3.66
23	58.57 ± 3.36	60.9 ± 3.87	72.16 ± 2.86	68.54 ± 4.36	66.93 ± 3.25	67.61 ± 2.85	71.65 ± 4.47
27	58.57 ± 2.85	62.99 ± 2.01	71.60 ± 3.47	66.23 ± 2.93	70.04 ± 1.95	71.88 ± 1.94	73.61 ± 3.56
31	58.04 ± 3.56	61.92 ± 2.25	71.02 ± 2.42	69.94 ± 2.43	70.50 ± 2.06	70.55 ± 1.93	72.31 ± 2.90
35	58.31 ± 2.31	62.89 ± 2.95	70.63 ± 2.35	67.54 ± 3.01	70.86 ± 2.12	70.93 ± 2.45	72.14 ± 2.05
39	58.90 ± 1.93	61.8 ± 1.64	70.65 ± 2.05	67.50 ± 2.18	70.81 ± 1.89	70.73 ± 1.94	71.69 ± 2.06
43	58.67 ± 1.53	61.74 ± 2.26	70.67 ± 1.57	68.62 ± 1.36	70.15 ± 1.75	70.31 ± 1.47	71.27 ± 1.53
47	58.40 ± 1.79	62.81 ± 1.57	71.43 ± 2.19	68.45 ± 1.11	69.45 ± 1.17	69.52 ± 1.32	71.40 ± 1.52
49	57.60 ± 1.33	62.78 ± 1.23	70.49 ± 1.03	68.21 ± 1.55	69.80 ± 1.28	69.85 ± 1.31	70.78 ± 1.53
50	57.63 ± 1.35	63.47 ± 1.37	70.79 ± 1.95	69.66 ± 1.17	69.83 ± 1.09	70.02 ± 0.93	73.00 ± 1.15
Avg.	58.79	60.92	71.44	67.78	68.84	69.52	72.74

presents the corresponding NMI results. Similarly, results for the COIL20 dataset are detailed in

Table 6. AC (%) of each algorithm on Coil20 dataset.

Classes	NMF	NTD	GNTD	GDNMF	GDNMTF	GSNMTF	GDSNTD
2	97.92 ± 4.21	76.93 ± 14.22	1 ± 0	1 ± 0	86.49 ± 17.62	99.86 ± 0.28	1 ± 0
4	79.26 ± 16.73	79.07 ± 15.15	93.65 ± 11.09	85.87 ± 11.42	74.72 ± 15.61	77.62 ± 16.93	95.40 ± 8.99
6	67.24 ± 12.00	65.14 ± 12.03	86.75 ± 9.76	78.11 ± 13.43	81.63 ± 11.37	82.63 ± 11.34	83.73 ± 15.22
8	72.17 ± 7.71	65.94 ± 6.14	85.94 ± 7.76	86.87 ± 9.70	88.00 ± 8.82	88.90 ± 9.13	86.32 ± 6.78
10	67.52 ± 5.09	69.11 ± 4.78	87.62 ± 8.32	86.64 ± 7.76	85.75 ± 8.30	85.93 ± 7.90	88.74 ± 9.01
12	66.43 ± 7.92	64.14 ± 5.07	83.05 ± 7.79	78.69 ± 8.22	78.40 ± 4.71	79.42 ± 6.73	86.39 ± 7.22
14	64.58 ± 5.15	63.58 ± 5.98	79.30 ± 5.52	81.04 ± 8.16	82.28 ± 3.47	82.44 ± 4.95	81.37 ± 5.39
16	63.86 ± 6.14	60.67 ± 4.48	82.45 ± 4.37	78.67 ± 4.89	80.86 ± 6.45	80.96 ± 7.28	84.10 ± 3.12
18	61.71 ± 3.93	61.09 ± 4.28	79.12 ± 5.19	78.58 ± 4.31	79.59 ± 4.15	81.90 ± 4.87	81.02 ± 4.78
19	62.06 ± 3.79	59.67 ± 3.78	76.58 ± 4.00	76.32 ± 4.67	77.81 ± 3.44	78.09 ± 3.65	79.37 ± 2.90
20	61.19 ± 4.03	57.62 ± 4.15	78.06 ± 3.63	79.21 ± 3.62	78.95 ± 2.59	79.40 ± 3.05	79.85 ± 2.79
Avg.	69.45	65.72	84.77	82.73	84.17	83.38	86.03

(accuracy) and

Table 7. NMI (%) of each algorithm on Coil20 dataset.

Classes	NMF	NTD	GNTD	GDNMF	GDNMTF	GSNMTF	GDSNTD
2	91.99 ± 16.19	40.04 ± 30.52	93.35 ± 8.58	1 ± 0	66.48 ± 42.53	98.94 ± 2.14	1 ± 0
4	75.51 ± 18.55	75.94 ± 14.84	93.35 ± 9.47	81.75 ± 11.19	76.83 ± 16.39	74.81 ± 20.31	94.47 ± 8.55
6	65.27 ± 1.34	64.32 ± 11.79	86.17 ± 9.08	78.73 ± 12.42	83.13 ± 8.87	83.88 ± 9.88	85.39 ± 11.34
8	76.67 ± 6.90	70.71 ± 5.58	88.91 ± 4.81	89.55 ± 7.08	91.67 ± 6.04	92.18 ± 5.70	89.58 ± 4.03
10	74.23 ± 3.05	74.83 ± 4.02	91.71 ± 5.22	91.48 ± 4.60	90.51 ± 5.36	89.91 ± 5.75	92.22 ± 5.61
12	73.35 ± 6.11	71.96 ± 2.98	88.55 ± 5.99	86.72 ± 4.57	85.57 ± 3.70	86.17 ± 4.54	91.18 ± 5.31
14	74.63 ± 3.11	72.13 ± 5.24	86.35 ± 3.18	88.58 ± 4.94	90.34 ± 2.41	89.84 ± 3.06	87.51 ± 3.13
16	73.87 ± 3.70	71.73 ± 2.72	89.20 ± 2.31	87.10 ± 2.59	88.78 ± 4.06	88.38 ± 4.48	90.99 ± 1.88
18	72.77 ± 2.61	72.49 ± 2.84	86.78 ± 3.11	86.97 ± 2.13	89.78 ± 2.51	89.99 ± 2.87	89.10 ± 2.45
19	73.43 ± 2.32	72.17 ± 2.58	85.55 ± 1.94	86.44 ± 2.03	87.62 ± 1.39	87.6 ± 1.89	88.02 ± 1.90
20	72.99 ± 1.80	71.32 ± 2.22	85.97 ± 1.70	88.07 ± 1.46	88.15 ± 1.06	88.05 ± 1.14	88.33 ± 1.07
Avg.	74.97	68.88	89.32	87.76	89.60	88.16	90.59

(NMI). From the results, we derive the following main conclusions:

1.

From Table 2, we can observe that the value of AC of GDSNTD is better than that of other methods on most datasets. The value of NMI reflects the proportion of correct decisions, which also proves that the proposed model performs better. The performance improvement is evident in the Georgia dataset, where GDSNTD improves the accuracy by 48.92% over the NMF co-clustering algorithm. The accuracy of GDSNTD is also improved by 26.1%, 5.15%, 7.02%, 3.71%, and 2.82% compared with other co-clustering algorithms NTD, GNTD, GDNMF, GDNMTF, and GSNMTF, respectively. In the Coil20 dataset, the accuracy of GDSNTD is also improved by 30.50%, 38.58%, 2.29%, 0.81%, 1.14%, and 0.57%, compared with other co-clustering algorithms NTD, GNTD, GDNMF, GDNMTF, and GSNMTF, respectively. In the Iris dataset, the accuracy of GDSNTD is improved by 17.22%, 32.77%, 2.84%, 2.10%, 0.17%, and 0.12% compared with other co-clustering algorithms NMF, NTD, GNTD, GDNMF, GDNMTF, and GSNMTF, respectively.

2.

From Table 3, we can observe that our proposed method of GDSNTD can also achieve higher accuracy. The NMI of GDSNTD is as high as 50.16% on the Iris dataset, which is 34.95%, 7.46%, 4.27%, 3.49%, and 3.42% better compared with NMF, GNTD, GDNMF, GDNMTF, and GSNMTF, respectively. In the Georgia dataset, the accuracy of GDSNTD is also improved by 26.67%, 15.01%, 3.12%, 4.79%, 4.54%, and 4.26% compared with other co-clustering algorithms NMF, NTD, GNTD, GDNMF, GDNMTF, and GSNMTF, respectively. In the Coil20 dataset, the accuracy of GDSNTD is improved by 21.02%, 23.85%, 2.75%, 0.3%, 0.2%, and 0.32%, compared with other co-clustering algorithms NMF, NTD, GNTD, GDNMF, GDNMTF, and GSNMTF, respectively.

3.

Table 4 shows that the average AC of GDSNTD is higher than that of the other methods on most datasets. In the Georgia dataset, the accuracy of the co-clustering algorithm GDSNTD is improved by an average of 27.4%, 22.6%, 2.2%, 8.7%, 5.11%, and 3.79% compared with NMF, NTD, GNTD, GDNMF, GDNMTF, and GSNMTF, respectively. From Table 6, it can be observed that on the Coil20 dataset, the accuracy of GDSNTD is, on average, 23.9%, 30.9%, 1.49%, 4.0%, 5.79%, and 3.18% higher than that of NMF, NTD, GNTD, GDNMF, GDNMTF, and GSNMTF, respectively.

4.

From Table 5, it can be observed that on the Georgia dataset, the accuracy of GDSNTD is, on average, 23.7%, 19.4%, 1.82%, 7.3%, 5.67%, and 4.63% higher than that of NMF, NTD, GNTD, GDNMF, GDNMTF, and GSNMTF, respectively. From Table 7, it can be observed that on the Coil20 dataset, the accuracy of GDSNTD is, on average, 20.8%, 31.5%, 1.42%, 3.2%, 6.14%, and 2.76% higher than that of NMF, NTD, GNTD, GDNMF, GDNMTF, and GSNMTF, respectively.

Figure 3 shows the clustering performance on the Georgia, COIL20, and Iris datasets. It can be observed that the proposed GDSNTD method surpasses all other compared methods.

4.3. Interpretation of GDSNTD’s Superior Performance

Across all datasets, our proposed method consistently outperforms all competing baselines of NMF, NTD, GNTD, GDNMF, GDNMTF, and GSNMTF in both standard clustering and co-clustering tasks. This empirical evidence strongly suggests that the GDSNTD model with dual graph constraints successfully leads to a more structured and discriminative latent space, thereby improving clustering accuracy.

First, applying manifold constraints to the factor matrices is equivalent to preserving the structure in the “essential features” of the data, resulting in greater accuracy. Second, $l_1$ regularization counterbalances the Frobenius norm, leading to a more stable optimization process and yielding factors within a more reasonable numerical range. Furthermore, $l_1$ regularization promotes the learning of “parts” that correspond to local structures within the data, thereby promoting a decomposition with enhanced coherence.

4.4. Parameter Selection

Every parameter is searched over a range of values from 0 to ${10}^5$. The choice of parameters has a pronounced effect on experimental performance; therefore, they were selected systematically through a grid search. Figure 4 shows the process of determining the optimal parameters using a grid search on the Iris dataset. From Figure 4, we can see that the accuracy is only $0.7772$ when ${\alpha }_V={\alpha }_W={10}^0$ and ${\gamma }_V={\gamma }_W={10}^0$. When the coefficient of the sparse regularization term is ${\gamma }_V={\gamma }_W={10}^2$, the accuracy improves to $0.7980$. From Figure 5, it is to be observed that the NMI is $0.8615$ when ${\alpha }_V={\alpha }_W={10}^{-3}$ and ${\gamma }_V={\gamma }_W={10}^1$, and when the coefficient of the sparse regularization term is ${\gamma }_V={\gamma }_W={10}^{-2}$, the accuracy is only $0.8776$.

4.5. Convergence Study

As described in Section 3, the convergence of the proposed algorithms has been theoretically proved. In this subsection, we experimentally analyze the convergence of the proposed algorithms by examining the relationship between the number of iterations and the value of the objective function. This relationship is visualized in Figure 6, Figure 7 and Figure 8. The convergence behavior of GSNTD on the Georgia dataset is illustrated in Figure 6. The convergence behavior of GSNTD on the Coil20 dataset is illustrated in Figure 7. The convergence behavior of GSNTD on the Iris dataset is illustrated in Figure 8. The observed monotonic decrease in the objective function value demonstrates that the algorithm converges effectively under the multiplicative update rules. This result provides empirical support for the convergence proof given in Theorem 1. We can find that GDSNTD is usually able to reach convergence within 1000 iterations.

4.6. Complexity Analysis

In this subsection, we analyze the computational complexity of GDSNTD. Note that $\mathcal{X}$ is a third-order $I_1\times I_2\times I_3$-dimensional tensor, and $\mathcal{G}$ is a third-order $J_1\times J_2\times J_3$-dimensional core tensor. Factor matrices are $\mathbf{U}\in {\mathbb{R}}_{+}^{I_1\times J_1}$, $\mathbf{V}\in {\mathbb{R}}_{+}^{I_2\times J_2}$, and $\mathbf{W}\in {\mathbb{R}}_{+}^{I_3\times J_3}$.

Consider the updating rule in (8), the operation ${(\mathbf{W}\otimes \mathbf{V}\otimes \mathbf{U})}^{\top }vec\left(\mathcal{X}\right)$ corresponds to the tensor mode product $\mathcal{X}{\times }_1{\mathbf{U}}^{\top }{\times }_2{\mathbf{V}}^{\top }{\times }_3{\mathbf{W}}^{\top }$. Its computational complexity is $\mathcal{O}(I_1{I}_2{I}_3{J}_1+I_2{I}_3{J}_1{J}_2+I_3{J}_1{J}_2{J}_3)$. ${(\mathbf{W}\otimes \mathbf{V}\otimes \mathbf{U})}^{\top }(\mathbf{W}\otimes \mathbf{V}\otimes \mathbf{U})vec\left(\mathcal{G}\right)$ corresponds to $vec\left(\mathcal{G}{\times }_1{\mathbf{U}}^{\top }\mathbf{U}{\times }_2{\mathbf{V}}^{\top }\mathbf{V}{\times }_3{\mathbf{W}}^{\top }\mathbf{W}\right)$. The computational complexity is $J_1{J}_2{J}_3(J_1+J_2+J_3)$. The total computational complexity of computing the update of $\mathcal{G}$ is bounded by $\mathcal{O}(I_1{I}_2{I}_3{J}_1+I_2{I}_3{J}_1{J}_2+I_3{J}_1{J}_2{J}_3+J_1{J}_2{J}_3(J_1+J_2+J_3))$.

Consider the updating rule in (13), ${\mathbf{X}}_{\left(1\right)}(\mathbf{W}\otimes \mathbf{V}){\mathbf{G}}_{\left(1\right)}^{\top }$ corresponds to ${\left(\mathcal{X}{\times }_2{\mathbf{V}}^{\top }{\times }_3{\mathbf{W}}^{\top }\right)}_{\left(1\right)}{\mathbf{G}}_{\left(1\right)}^{\top }$, which needs $\mathcal{O}(I_1{I}_2{I}_3{J}_2+I_1{I}_3{J}_2{J}_3+I_1{J}_1{J}_2{J}_3)$ operations. $\mathbf{U}{\mathbf{G}}_{\left(1\right)}{(\left(\mathbf{W}\otimes \mathbf{V}\right)}^{\top }$$(\mathbf{W}\otimes \mathbf{V})){\mathbf{G}}_{\left(1\right)}^{\top }$ corresponds to ${\left(\mathcal{G}{\times }_1\mathbf{U}{\times }_2{\mathbf{V}}^{\top }\mathbf{V}{\times }_3{\mathbf{W}}^{\top }\mathbf{W}\right)}_{\left(1\right)}{\mathbf{G}}_{\left(1\right)}^{\top }$, which needs $\mathcal{O}(J_1^2{J}_2{J}_3+J_1{J}_2^2{J}_3+J_1{J}_2{J}_3^2+I_1{J}_1^2)$ operations. The total computational complexity of computing the update of $\mathbf{U}$ is bounded by $\mathcal{O}(I_1{I}_2{I}_3{J}_2+I_1{I}_3{J}_2{J}_3+I_1{J}_1{J}_2{J}_3+J_1^2{J}_2{J}_3+J_1{J}_2^2{J}_3+J_1{J}_2{J}_3^2+I_1{J}_1^2)$.

Consider the updating rule in (23), ${{\mathbf{X}}_{\left(2\right)}}^{\left(t\right)}({\mathbf{W}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)}){{\mathbf{G}}_{\left(2\right)}^{\left(t\right)}}^{\top }+{\lambda }_V{\mathbf{C}}_{\mathbf{V}}^{\left(t\right)}{\mathbf{V}}^{\left(t\right)}$ corresponds to ${\left(\mathcal{X}{\times }_1{\mathbf{U}}^{\top }{\times }_3{\mathbf{W}}^{\top }\right)}_{\left(2\right)}{\mathbf{G}}_{\left(2\right)}^{\top }+{\lambda }_V{\mathbf{C}}_{\mathbf{V}}\mathbf{V}$, and it takes $\mathcal{O}(I_1{I}_2{I}_3{J}_1+I_2{I}_3{J}_1{J}_3+I_2{J}_1{J}_2{J}_3+I_2^2{J}_2)$ operations. ${\mathbf{V}}^{\left(t\right)}{\mathbf{G}}_{\left(2\right)}^{\left(t\right)}{({\mathbf{W}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)})}^{\top }({\mathbf{W}}^{\left(t\right)}\otimes {\mathbf{U}}^{\left(t\right)}){{\mathbf{G}}_{\left(2\right)}^{\left(t\right)}}^{\top }+{\alpha }_V{\mathbf{V}}^{\left(t\right)}+{\lambda }_V{\mathbf{D}}_{\mathbf{V}}^{\left(t\right)}$${\mathbf{V}}^{\left(t\right)}$ corresponds to ${\left(\mathcal{G}{\times }_1{\mathbf{U}}^{\top }\mathbf{U}{\times }_2\mathbf{V}{\times }_3{\mathbf{W}}^{\top }\mathbf{W}\right)}_{\left(2\right)}{\mathbf{G}}_{\left(2\right)}^{\top }+{\alpha }_V\mathbf{V}+{\lambda }_V{\mathbf{D}}_{\mathbf{V}}\mathbf{V}$, and it takes $\mathcal{O}(J_1^2{J}_2{J}_3+J_1{J}_2^2{J}_3+J_1{J}_2{J}_3^2+I_2{J}_2^2+I_2^2{J}_2+I_2{J}_2)$ operations. The total computational complexity of computing the update of $\mathbf{V}$ is bounded by $\mathcal{O}(I_1{I}_2{I}_3{J}_1+I_2{I}_3{J}_1{J}_3+I_2{J}_1{J}_2{J}_3+2I_2^2{J}_2+J_1^2{J}_2{J}_3+J_1{J}_2^2{J}_3+J_1{J}_2{J}_3^2+I_2{J}_2^2+I_2{J}_2)$.

Similarly, consider the updating rule in (34), the total computational complexity of computing the update of $\mathbf{W}$ is bounded by $\mathcal{O}(I_1{I}_2{I}_3{J}_1+I_2{I}_3{J}_1{J}_2+I_3{J}_1{J}_2{J}_3+2I_3^2{J}_3+J_1^2{J}_2{J}_3+J_1{J}_2^2{J}_3+J_1{J}_2{J}_3^2+I_3{J}_3^2+I_3{J}_3)$.

Therefore, the total computational complexity of the proposed method is $\mathcal{O}(I_1{I}_2{I}_3(3J_1+J_2)+I_2{I}_3(2J_2+J_3)+I_1{I}_3{J}_2{J}_3+J_1{J}_2{J}_3(2I_2+I_1+I_2+4(J_1+J_2+J_3))+2I_2^2{J}_2+2I_3^2{J}_3+I_1{J}_1^2+I_2{J}_2^2+I_3{J}_3^2+I_2{J}_2+I_3{J}_3)$. Compared with GNTD, the total computational complexity of the GDSNTD is increased by $2I_2^2{J}_2+2I_3^2{J}_3+I_2{J}_2+I_3{J}_3$.

Figure 9 shows the relative performance in terms of clustering ACC and time consumption among NTD, GNTD, GDNMTF, GSNMTF, and GDSNTD on the Georgia dataset. Figure 10 shows the relative performance in terms of clustering NMI and time consumption among NTD, GNTD, GDNMTF, GSNMTF, and GDSNTD on the Coil20 dataset. In the figure, CPU(s) is the running time in seconds.

5. Conclusions

In this paper, we propose a co-clustering method via a dual graph-regularized sparse non-negative Tucker decomposition (GDSNTD) framework. It incorporates the Frobenius norm, graph regularization, and an $l_1$ norm constraint. The graph regularization term is used to maintain the intrinsic geometric structure of the data. This approach allows the model to represent the internal structure of high-dimensional data more accurately. The model not only adds the $l_1$ norm to achieve the adjustment of data eigenvalues but also embeds the Frobenius norm to improve the generalization ability of the model. We then present a detailed derivation of the solution, design the accompanying algorithm for GDSNTD, and provide a convergence proof. The experimental results show that selecting appropriate parameters can improve the model’s classification accuracy. This indicates that the selected rules have practical significance. Experiments on public datasets demonstrate that our proposed model achieves superior image clustering performance. The proposed method outperforms the state-of-the-art method GSNMTF by an average of 3.79% in clustering accuracy and 4.63% in Normalized Mutual Information on the Georgia dataset.

6. Future Work

Although the proposed GSNTTD method has shown good clustering performance, it operates on the assumption that the data resides in a well-formed latent vector space [34]. In recent years, deep learning-based clustering has experienced rapid advancement. The current popular deep learning-based methods are dominated by several key approaches, such as Graph Autoencoders (GAEs), graph convolutional network-based clustering (GCN-based clustering), and Deep Tensor Factorization.

GAEs aim to learn low-dimensional latent representations (embeddings) of graph-structured data in an unsupervised manner. It learns to compress input data into a lower-dimensional latent representation (encoding) and then reconstruct the original input from this representation (decoding). The critical distinction is that the input data is a graph, characterized by its topology (structure) and node attributes (features). Kipf et al. in [35] formally introduced the GAE and its variational counterpart, the variational graph autoencoder (VGAE). The field of GAE has evolved rapidly from foundational models to highly sophisticated and specialized architectures. Recent advancements have focused on developing more sophisticated architectures to address specific challenges. Zhou K et al. proposed a new causal representation method based on a graph autoencoder embedded autoencoder (GeAE). The GeAE employs a causal structure learning module to account for non-linear causal relationships present in the data [36].

Kipf and Welling in [35] introduced the variational graph autoencoder (VGAE), a framework for unsupervised learning on graph-structured data based on the variational autoencoder (VAE). In this work, they demonstrated this VGAE model using a GCN encoder and a simple inner product decoder. In [37], Kipf and Welling first introduced the efficient, first-order approximation-based graph convolutional layer. This formulation has laid the groundwork for numerous subsequent GCN studies. The core idea of a GCN encoder is to learn low-dimensional vector representations (i.e., node embeddings) for nodes by propagating and transforming information across the graph structure. The node’s embedding is determined not only by its own features but also jointly by the features of its neighbor nodes and the local graph structure. The core principle of GCN-based clustering is to unify node representation learning and cluster assignment into an end-to-end, jointly optimized framework and perform these tasks simultaneously.

The principle of Deep Tensor Factorization can be understood as using a deep neural network to perform the factorization and reconstruction process. Wu et al. proposed a Neural Tensor Factorization model, which incorporates a multi-layer perceptron structure to learn the non-linearities between different latent factors [38]. In [39], Jiang et al. proposed a generic architecture of deep transfer tensor factorization (DTTF), where the side information is embedded to provide effective compensation for the tensor sparsity.

In [40], Ballard et al. categorized feedforward neural networks, graph convolutional neural networks, and autoencoders into non-generative deep learning-based multi-omics integration methods. They found that deep learning-based approaches build off of previous statistical methods to integrate multi-omics data by enabling the modeling of complex and non-linear interactions between data types. Therefore, deep learning-based clustering has emerged as a rapidly advancing field. It integrates conventional cluster analysis with deep learning, leveraging its powerful feature representation and non-linear mapping capabilities to demonstrate superior performance in unsupervised clustering tasks. In the future, we intend to incorporate recent deep learning-based methods into our subsequent research to explore more up-to-date clustering methodologies.