Autoencoder-like Sparse Non-Negative Matrix Factorization with Structure Relationship Preservation

Abstract

Clustering algorithms based on non-negative matrix factorization (NMF) have garnered significant attention in data mining due to their strong interpretability and computational simplicity. However, traditional NMF often struggles to effectively capture and preserve topological structure information between data during low-dimensional representation. Therefore, this paper proposes an autoencoder-like sparse non-negative matrix factorization with structure relationship preservation (ASNMF-SRP). Firstly, drawing on the principle of autoencoders, a “decoder-encoder” co-optimization matrix factorization framework is constructed to enhance the factorization stability and representation capability of the coefficient matrix. Then, a preference-adjusted random walk strategy is introduced to capture higher-order neighborhood relationships between samples, encoding multi-order topological structure information of the data through an optimal graph regularization term. Simultaneously, to mitigate the impact of noise and outliers, the $l_{2,1}$-norm is used to constrain the feature correlation between low-dimensional representations and the original data, preserving feature relationships between data, and a sparse constraint is imposed on the coefficient matrix via the inner product. Finally, clustering experiments conducted on 8 public datasets demonstrate that ASNMF-SRP consistently exhibits favorable clustering performance.

1. Introduction

Clustering aims to reveal intrinsic relationships between data through similarity measurement, and it is widely applied in various fields such as marketing [1], gene expression [2], and pattern recognition [3]. However, with the continuous increase in data scale, traditional clustering algorithms face challenges posed by the “curse of dimensionality”, including increased computational complexity, feature redundancy, and amplified noise. Consequently, researchers have proposed a collaborative optimization paradigm of “dimensionality reduction followed by clustering”. Matrix factorization, as a crucial dimensionality reduction method, factorizes the original data matrix into low-rank submatrices to achieve the goal of “parts representing the whole”. Common matrix factorization methods include Principal Component Analysis (PCA) [4], Singular Value Decomposition (SVD) [5], and non-negative matrix factorization (NMF) [6]. Due to advantages such as non-negativity constraints, computational simplicity, and fast implementation, NMF is frequently used in clustering analysis.

In recent years, to further enhance the performance of NMF, researchers have proposed many improved NMF algorithms. For example, Hoyer et al. [7] proposed sparse non-negative matrix factorization (SNMF), utilizing $l_1$-norm to improve the sparsity of NMF. Kong et al. [8] proposed robust non-negative matrix factorization (RNMF), replacing the Frobenius norm with the $l_{2,1}$-norm to enhance the algorithm’s robustness. Ding et al. [9] proposed orthogonal non-negative matrix factorization (ONMF), reducing interference from redundant information by imposing orthogonality constraints on the factorized submatrices. These NMF algorithms primarily focus on the reconstruction error after matrix factorization but overlook the spatial structural relationships within the original data. Studies have shown that introducing graph regularization techniques into NMF can not only optimize reconstruction error but also further capture local geometric structures between data [10]. Cai et al. [11] proposed the graph regularized non-negative matrix factorization algorithm (GNMF), encoding topological information between data into the objective function to effectively improve clustering performance. Wu et al. [12] proposed robust manifold non-negative matrix factorization (MNMEL21), which considers manifold learning of data and employs the l_2,1-norm to mitigate interference from noise and outliers. Li et al. [13] proposed graph regularized non-negative low-rank matrix factorization (GNLMF), enhancing clustering accuracy on image data by capturing low-rank structures. Liu et al. [14] proposed graph regularized discriminative non-negative matrix factorization (GDNMF), which incorporates local geometric structure information between samples and data label information to effectively improve clustering performance on image data. To enhance the quality of graph learning, Huang et al. [15] proposed non-negative matrix factorization with adaptive graph learning (NMFAN), dynamically adjusting graph structure learning to effectively improve algorithm performance. Ren et al. [16] proposed semi-supervised symmetric non-negative matrix factorization with graph quality improvement (S³NMFGC), improving clustering performance by dynamically generating and adaptively updating graph learning results. Mohammadi et al. [17] proposed a semi-supervised multi-view clustering method based on adaptive symmetric NMF (SSA-SNMF), which improves the performance of the algorithm by introducing a multi-constraint optimization strategy.

In the process of graph learning, the quality of the learned graph structure directly affects the accuracy and robustness of clustering results. Currently, most graph learning methods are based on measuring Euclidean distances between samples. As the complexity of data internal structures increases, some samples may exhibit distant geometric distances, yet their local neighborhood topologies might show highly similar characteristics. Therefore, with the development of Graph Convolutional Neural Networks (GCNNs) [18], higher-order topological relationships between samples have gained widespread attention among researchers. Wang et al. [19] proposed a robust high-order graph learning algorithm for multi-view clustering (RHGL), which improves clustering accuracy by learning high-order graphs. Zhan et al. [20] proposed a multi-view clustering method with optimal high-order graph embedding (Co-MSE), which enhances the quality of graph learning by modeling high-order correlations. Additionally, Wang et al. [21] indicated that autoencoder-based methods can not only reduce data dimensionality but also further learn latent information between data points. Thus, from the perspective of “decoder-encoder” collaborative optimization, although the aforementioned improved NMF-based algorithms enhance clustering performance through different regularization terms, their objective functions only include the “decoder” component. They lack an explicit “encoder” structure with constraints on the coefficient matrix, thus preventing the coefficient matrix from fully uncovering latent information in the raw data.

Research indicates that introducing sparsity constraints into NMF can effectively enhance the clustering performance of algorithms [7,22]. Meng et al. [23] proposed sparse and orthogonally constrained dual-graph regularized non-negative matrix factorization (SODNMF), which incorporates dual-graph regularization, sparsity constraints, and orthogonality constraints into the objective function, significantly improving the algorithm’s performance. Peng et al. [24] proposed logarithmic sparse non-negative matrix factorization (LS-NMF), designing a $l_{\log }$-(pseudo) norm to achieve sparse constraints. Xiong et al. [25] proposed dual-graph regularized sparse robust and adaptive non-negative matrix factorization (DRGSNMF), adopting the $L_{2,p}$-norm to construct a sparse regularization term. Although the implementations of these algorithms differ, they all further enhance algorithm performance through sparse constraints.

Features often exhibit complex correlations or complementarity, such as in consumer behavior data. Therefore, in NMF, optimizing only the coefficient matrix may fail to effectively preserve structural relationships between features. To address this, Shang et al. [26] proposed dual-graph regularized non-negative matrix factorization (DNMF), which enhances the algorithm’s generalization capability by incorporating learning of the feature graph. Gu et al. [27] proposed a dual-graph regularized co-optimization clustering algorithm (DRCC), simultaneously considering the learning of both sample and feature graphs. Furthermore, to ensure the effective preservation of feature relationships from the original data matrix during dimensionality reduction, Hedjam et al. [28] proposed feature relationship-preserving non-negative matrix factorization (FR-NMF), and Salahian et al. [29] proposed a deep autoencoder non-negative matrix factorization with contrastive regularization and feature relationship preservation (DANMF-CRFR).

Therefore, this paper proposes an autoencoder-like sparse non-negative matrix factorization with structure relationship preservation (ASNMF-SRP). This method incorporates autoencoder principles into NMF, extending the constraints on the coefficient matrix from implicit to explicit. Through these dual constraints, it not only enhances the decomposition stability of the coefficient matrix but also further uncovers latent information in the raw data. Simultaneously, it preserves topological information in the original data space using higher-order graph regularization terms for both samples and features and incorporates an optimization term for feature relationship preservation. Finally, sparse optimization is applied to the coefficient matrix. Thus, ASNMF-SRP integrates autoencoder-like NMF, higher-order graph regularization, feature relationship preservation, and sparse constraints into a unified optimization framework.

The main contributions of the ASNMF-SRP algorithm are as follows:

(1) Optimization of the coefficient matrix. Firstly, this paper introduces an autoencoder-like NMF, adding an explicit “encoder” structure to constrain the coefficient matrix. Then, in the local structure learning process, a higher-order graph regularization method is proposed, enabling the algorithm to progressively extend from first-order to higher-order graph learning. Finally, an inner-product-based sparse constraint is introduced, preserving high-order collaborative relationships between data at the element level.

(2) Optimization of the basis matrix. This paper proposes a robust feature relationship preservation method, thereby establishing a topology-preserving relationship between the low-dimensional embedding space and the original high-dimensional data feature structure.

(3) This paper demonstrates the feasibility of ASNMF-SRP through convergence analysis, time complexity analysis, and other aspects. Simultaneously, in the experimental section, comparative analysis of clustering results, visualization analysis of low-dimensional representation data, parameter sensitivity analysis, and other experiments are conducted. Ultimately, the experimental results show that ASNMF-SRP exhibits excellent clustering performance.

The remaining work in this paper is organized as follows: Section 2 introduces some foundational work related to ASNMF-SRP. Section 3 primarily describes the derivation of the ASNMF-SRP algorithm, including the convergence and time complexity of submatrix update iterations. Section 4 presents a series of experiments conducted on eight public datasets. Section 5 provides a summary of the ASNMF-SRP algorithm.

2. Related Work

2.1. Non-Negative Matrix Factorization (NMF)

The objective of NMF is to factorize a given non-negative data matrix $\mathit{X}=\left({\mathit{x}}_1,{\mathit{x}}_2,\cdots ,{\mathit{x}}_n\right)\in {ℝ}_{+}^{m\times n}$ into the product of two non-negative and low-rank matrices: a basis matrix $\mathit{U}\in {ℝ}_{+}^{m\times r}$ and a coefficient matrix $\mathit{V}\in {ℝ}_{+}^{r\times n}$, i.e., $\mathit{X}\approx \mathit{U}\mathit{V}$, where $r\ll \min \left\{m,n\right\}$. During the factorization process, NMF optimizes the approximate representation by minimizing the reconstruction error. Therefore, it employs the Frobenius norm to define the objective function as follows:

$$\min {‖\mathit{X}-\mathit{U}\mathit{V}‖}_F^2,\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\mathrm{s}.\mathrm{t}.\hspace{0.33em}\mathit{U}\ge 0,\hspace{0.33em}\mathit{V}\ge 0$$

(1)

Here, ${‖\cdot ‖}_F$ denotes the Frobenius norm of a matrix, which measures the reconstruction error after matrix factorization. For the optimization problem in Equation (1), Lee et al. [30] proposed an iterative algorithm based on multiplicative update rules, with the update formulas given by the following:

$${\mathit{U}}_{ij}\leftarrow {\mathit{U}}_{ij}{\displaystyle \frac{{\left({\mathit{X}\mathit{V}}^T\right)}_{ij}}{{\left({\mathit{U}\mathit{V}\mathit{V}}^T\right)}_{ij}}},\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}{\mathit{V}}_{ij}\leftarrow {\mathit{V}}_{ij}{\displaystyle \frac{{\left({\mathit{U}}^T\mathit{X}\right)}_{ij}}{{\left({\mathit{U}}^T\mathit{U}\mathit{V}\right)}_{ij}}}$$

(2)

2.2. Robust Non-Negative Matrix Factorization (RNMF)

NMF measures reconstruction error based on Euclidean distance, making it highly susceptible to noise and outliers. To further enhance the robustness of the matrix factorization process, Kong et al. [8] proposed robust non-negative matrix factorization (RNMF), which employs the $l_{2,1}$-norm to assess the quality of data reconstruction. The objective function of RNMF is as follows:

$$\min {‖\mathit{X}-\mathit{U}\mathit{V}‖}_{2,1},\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\mathrm{s}.t.\hspace{0.33em}\mathit{U}\ge 0,\hspace{0.33em}\mathit{V}\ge 0$$

(3)

The update formulas for the basis matrix $\mathit{U}$ and coefficient matrix $\mathit{V}$ in Equation (3) are as follows:

$${\mathit{U}}_{ij}\leftarrow {\mathit{U}}_{ij}{\displaystyle \frac{{\left(\mathit{X}{\mathit{G}}_1{\mathit{V}}^T\right)}_{ij}}{{\left(\mathit{U}\mathit{V}{\mathit{G}}_1{\mathit{V}}^T\right)}_{ij}}},\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}{\mathit{V}}_{ij}\leftarrow {\mathit{V}}_{ij}{\displaystyle \frac{{\left({\mathit{U}}^T\mathit{X}{\mathit{G}}_1\right)}_{ij}}{{\left({\mathit{U}}^T\mathit{U}\mathit{V}{\mathit{G}}_1\right)}_{ij}}}$$

(4)

Here, G₁ is a diagonal matrix with diagonal elements given by the following:

$${\left({\mathit{G}}_1\right)}_{jj}=1/{‖{\left(\mathit{X}-\mathit{U}\mathit{V}\right)}_i‖}_2$$

(5)

2.3. Graph Regularized Non-Negative Matrix Factorization (GNMF)

The standard NMF achieves the goal of “representing the whole by parts” by minimizing the reconstruction error, but this approach overlooks the latent structural information in the data. Therefore, Cai et al. [11] proposed graph regularized non-negative matrix factorization (GNMF). GNMF improves upon NMF by encoding the local geometric structure among samples into the objective function for optimization, thereby enhancing the representation quality of samples in the low-dimensional space. The objective function of GNMF is as follows:

$$\min {‖\mathit{X}-\mathit{U}\mathit{V}‖}_F^2+\lambda \mathrm{Tr}\left({\mathit{V}\mathit{L}\mathit{V}}^T\right),\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\mathrm{s}.\mathrm{t}.\hspace{0.33em}\mathit{U}\ge 0,\hspace{0.33em}\mathit{V}\ge 0$$

(6)

Here, $\lambda$ is a non-negative regularization parameter that adjusts the strength of graph learning. $\mathit{L}$ is the Laplacian matrix, which satisfies $\mathit{L}=\mathit{D}-\mathit{W}$. In this context, $\mathit{W}$ is the similarity matrix reflecting the relationships between samples, and the degree matrix $\mathit{D}=\mathrm{diag}({\mathit{D}}_{11},{\mathit{D}}_{22},\cdots ,{\mathit{D}}_{nn})$ is a diagonal matrix formed by the row sums of $\mathit{W}$, i.e., ${\mathit{D}}_{ii}={\displaystyle {\sum }_j{\mathit{W}}_{ij}}$. The update rules for GNMF are as follows:

$${\mathit{U}}_{ij}\leftarrow {\mathit{U}}_{ij}{\displaystyle \frac{{\left({\mathit{X}\mathit{V}}^T\right)}_{ij}}{{\left({\mathit{U}\mathit{V}\mathit{V}}^T\right)}_{ij}}},\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}{\mathit{V}}_{ij}\leftarrow {\mathit{V}}_{ij}{\displaystyle \frac{{\left({\mathit{U}}^T\mathit{X}+\lambda \mathit{W}\mathit{V}\right)}_{ij}}{{\left({\mathit{U}}^T\mathit{U}\mathit{V}+\lambda \mathit{D}\mathit{V}\right)}_{ij}}}$$

(7)

3. Methodology

3.1. Autoencoder-like Non-Negative Matrix Factorization

In clustering analysis, the coefficient matrix $\mathit{V}$ of NMF serves as the clustering indicator matrix. Therefore, the coefficient matrix $\mathit{V}$ needs to effectively encode the structural features of the original data to ensure the accuracy of subsequent clustering. Inspired by the non-negative symmetric encoder-decoder method proposed by Sun et al. [31] in community detection and the principle of autoencoders, this paper imposes an explicit constraint on the coefficient matrix $\mathit{V}$, i.e., $\mathit{V}\approx f\left(\mathit{X}\right)={\mathit{U}}^T\mathit{X}$. This approach not only improves the factorization stability of the coefficient matrix but also further uncovers the latent representations of the original data. The specific implementation process is as follows:

Linear Decoder:

$$g\left(\mathit{v}\right)=\mathit{U}\mathit{v}\to \mathit{X}\approx g\left(\mathit{V}\right)=\mathit{U}\mathit{V}\to {‖\mathit{X}-\mathit{U}\mathit{V}‖}_F^2$$

(8)

Linear Encoder:

$$f\left(\mathit{x}\right)={\mathit{U}}^T\mathit{x}\to \mathit{V}\approx f\left(\mathit{X}\right)={\mathit{U}}^T\mathit{X}\to {‖\mathit{V}-{\mathit{U}}^T\mathit{X}‖}_F^2$$

(9)

In the “decoder-encoder” architecture, the coefficient matrix $\mathit{V}$ acts as the encoder matrix. From Equations (8) and (9), it can be seen that the “decoder” part corresponds to standard NMF, aiming to minimize the reconstruction error between data, i.e., maximizing the approximation $\mathit{X}\approx \mathit{U}\mathit{V}$. The “encoder” part, on the other hand, transforms the data matrix $\mathit{X}$ into a distributed representation using the basis matrix $\mathit{U}\in {ℝ}_{+}^{m\times r}$, thereby imposing an explicit constraint on the coefficient matrix, i.e., $\mathit{V}\approx {\mathit{U}}^T\mathit{X}$.

By combining Equations (8) and (9), an autoencoder-like non-negative matrix factorization framework is obtained, with the following objective function:

$$\min \underset{decoder}{\underbrace{{\left|\right|\mathit{X}-\mathit{U}\mathit{V}\left|\right|}_F^2}}+\underset{encoder}{\underbrace{{\left|\right|\mathit{V}-{\mathit{U}}^T\mathit{X}\left|\right|}_F^2}},\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\mathrm{s}.t.\hspace{0.33em}\mathit{U}\ge 0,\hspace{0.33em}\mathit{V}\ge 0$$

(10)

Here, the decoder and encoder parts share the same basis matrix $\mathit{U}$, thereby forming a symmetry constraint. Figure 1 is a schematic diagram of Equation (10).

3.2. High-Order Graph Regularization

To further improve the quality of the data representation in the low-dimensional space, this paper introduces graph regularization techniques based on Equation (10) to learn the geometric structural information of the original high-dimensional space. Therefore, this paper captures the neighborhood relationships between samples by constructing a nearest neighbor graph. The first-order similarity matrix ${\mathit{W}}_1$ is defined as follows:

$${\left({\mathit{W}}_1\right)}_{ij}=\left\{\begin{array}{c}1,\hspace{1em}{\mathit{x}}_i\in N_k\left({\mathit{x}}_j\right)\hspace{0.33em}\mathit{or}\hspace{0.33em}{\mathit{x}}_j\in N_k\left({\mathit{x}}_i\right)\hfill \\ 0,\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}otherwise\hfill \end{array}\right.$$

(11)

Here, $N_k\left({\mathit{x}}_i\right)$ denotes the set of $k$-nearest data points to sample ${\mathit{x}}_i$. ${\mathit{x}}_i$ and ${\mathit{x}}_j$ are the $i$-th and $j$-th column samples of the original data matrix $\mathit{X}=\left({\mathit{x}}_1,{\mathit{x}}_2,\cdots ,{\mathit{x}}_n\right)\in {ℝ}_{+}^{m\times n}$, respectively.

The coefficient matrix $\mathit{V}=\left({\mathit{v}}_1,{\mathit{v}}_2,\cdots ,{\mathit{v}}_n\right)\in {ℝ}_{+}^{r\times n}$ is the low-dimensional representation to be optimized. According to manifold learning and spectral graph theory, the smoothness of this low-dimensional representation is given by the following:

$$\begin{array}{l}\mathcal{Z}={\displaystyle \frac{1}{2}}{\displaystyle \sum _{i,j=1}^n{‖{\mathit{v}}_i-{\mathit{v}}_j‖}^2{\left({\mathit{W}}_1\right)}_{ij}}\\ \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}={\displaystyle \sum _{i=1}^n{\mathit{v}}_i{\mathit{v}}_i^T{\left({\mathit{D}}_1\right)}_{ii}}-{\displaystyle \sum _{i,j=1}^n{\mathit{v}}_i{\mathit{v}}_j^T{\left({\mathit{W}}_1\right)}_{ij}}\\ \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}=\mathrm{Tr}\left(\mathit{V}{\mathit{D}}_1{\mathit{V}}^T\right)-\mathrm{Tr}\left(\mathit{V}{\mathit{W}}_1{\mathit{V}}^T\right)\\ \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}=\mathrm{Tr}\left(\mathit{V}{\mathit{L}}_1{\mathit{V}}^T\right)\end{array}$$

(12)

Here, ${\mathit{D}}_1$ is a diagonal matrix with ${\left({\mathit{D}}_1\right)}_{ii}={\displaystyle {\sum }_j{\mathit{W}}_{ij}}$. ${\mathit{L}}_1$ is defined as the first-order Laplacian matrix, which satisfies the equation ${\mathit{L}}_1={\mathit{D}}_1-{\mathit{W}}_1$.

In real-world scenarios, samples often exhibit complex structural relationships. For instance, in online shopping, consumers from different regions may have no direct connection. However, by analyzing behavioral data, it is possible to identify neighborhood relationships where consumers share highly similar purchasing patterns. Such neighborhood relationships provide a scientific basis for businesses to segment target customer groups and implement precision marketing strategies. As shown in Figure 2, samples ${\mathit{x}}_i$ and ${\mathit{x}}_l$ are both connected to samples 1, 2, and 3, indicating that ${\mathit{x}}_i$ and ${\mathit{x}}_l$ share the same neighborhood structure. However, ${\mathit{x}}_i$ and ${\mathit{x}}_l$ are far apart and not within each other’s neighborhood sets. From Equation (11), it follows that ${\mathit{W}}_{il}=0$. Therefore, the first-order similarity matrix can only capture the neighborhood relationships between samples but fails to consider the similarity of their neighborhood structures. To further capture higher-order similarities between different neighborhood systems, this paper introduces a preference-adjusted random walk strategy. The random walk strategy [32,33] constructs sequences of sample points and encodes multi-step reachability into a higher-order similarity matrix, thereby reflecting higher-order structural information among samples. During the random walk process, the transition probability matrix ${\mathit{P}}_{ij}$ represents the probability of transitioning from sample ${\mathit{x}}_i$ to sample ${\mathit{x}}_j$, defined as follows:

$${\mathit{P}}_{ij}={\displaystyle \frac{a_{ij}}{k_i}}$$

(13)

Here, $a_{ij}$ indicates the connection state between samples ${\mathit{x}}_i$ and ${\mathit{x}}_j$: if ${\mathit{x}}_i$ and ${\mathit{x}}_j$ are connected, then $a_{ij}=1$; otherwise, $a_{ij}=0$. $k_i$ represents the degree of sample ${\mathit{x}}_i$. Since $a_{ij}$ takes values of 0 or 1, $k_i$ here denotes the number of samples connected to ${\mathit{x}}_i$.

The transition probability matrix in Equation (13) is based on an equal-probability assumption, which often fails to accurately reflect the topological heterogeneity among samples when processing high-dimensional complex data. Therefore, this paper incorporates the Preferential Attachment (PA) model into Equation (13). By assigning connection preference weights between sample points, the transition probability matrix is further optimized to more realistically reflect higher-order neighborhood relationships among samples. Thus, the new expression is as follows:

$${\mathit{P}}_{ij}={\displaystyle \frac{\mathrm{P}{\mathrm{A}}_{\mathrm{s}\mathrm{i}\mathrm{m}\mathrm{S}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{e}}}{\mathrm{M}\mathrm{a}\mathrm{x}\left(\mathrm{P}{\mathrm{A}}_{\mathrm{s}\mathrm{i}\mathrm{m}\mathrm{S}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{e}}\right)}}\times {\displaystyle \frac{a_{ij}}{k_i}}$$

(14)

Therefore, the $d$-th order similarity between samples ${\mathit{x}}_i$ and ${\mathit{x}}_j$ is as follows:

$${\mathit{S}}_{ij}^d={\mathit{P}}_{ik}\times {\mathit{P}}_{kl}\times \cdots \times {\mathit{P}}_{zj}$$

(15)

where $d=1,2,\cdots ,n$. In practical applications, the value of $d$ should not be too large, as an excessively large $d$ may introduce noise interference, thereby affecting the quality of graph learning. The average of the $d$-th order adjacency matrices is as follows:

$${\mathit{W}}_2=\left({\mathit{S}}^1+{\mathit{S}}^2+\cdots +{\mathit{S}}^d\right)/d$$

(16)

In Equation (16), ${\mathit{W}}_2$ is referred to as the second-order similarity matrix, which aims to capture higher-order neighborhood relationships between samples. The second-order Laplacian matrix is defined as ${\mathit{L}}_2={\mathit{D}}_2-{\mathit{W}}_2$, where ${\mathit{D}}_2$ is a diagonal matrix with entries ${\left({\mathit{D}}_2\right)}_{ii}={\displaystyle {\sum }_j{\mathit{W}}_{ij}}$.

By considering both the first-order and second-order Laplacian matrices, the optimal Laplacian matrix is now defined as follows:

$$\mathit{L}={\omega }_1{\mathit{L}}_1+{\omega }_2{\mathit{L}}_2$$

(17)

where $w_1$ and $w_2$ are balancing parameters. Consequently, the optimal graph regularization term is given by the following:

$$\underset{\mathit{V}\ge 0}{\min }\mathrm{Tr}\left(\mathit{V}\mathit{L}{\mathit{V}}^T\right)$$

(18)

3.3. Feature Relationship Preservation

As data structures increase in complexity, feature information also plays a crucial role in clustering analysis. For example, when enterprises identify target customer groups, consumer characteristics, such as purchase frequency (${\mathit{x}}_1^{*}$), interests (${\mathit{x}}_2^{*}$), spending power (${\mathit{x}}_3^{*}$), shopping preferences (${\mathit{x}}_4^{*}$), and brand loyalty (${\mathit{x}}_5^{*}$), are key dimensions. Therefore, when high-dimensional data are transformed into low-dimensional representations, preserving the intrinsic relationships between features becomes essential for effectively uncovering the latent structure of customer groups. This is illustrated in Figure 3.

In Figure 3, ${\mathit{x}}_i^{*}$ represents the i-th row vector of matrix $\mathit{X}$, corresponding to the i-th feature information in the original high-dimensional data, where $i=1,2,3,4,5$. $u_i^{*}$ represents the $i$-th row vector of the basis matrix $\mathit{U}$. As shown in Figure 3, the feature relationship between ${\mathit{x}}_i^{*}$ and $u_i^{*}$ remains consistent, which provides a theoretical basis for enterprises to identify target customer groups. Therefore, incorporating the feature relationships between data into the objective function for optimization can further preserve the feature relationships among the data.

${\left(\mathit{X}{\mathit{X}}^T\right)}_{ij}={\displaystyle {\sum }_{l=1}^n{\mathit{x}}_{il}{\mathit{x}}_{jl}}$ represents the inner product between the $i$-th and $j$-th features of $\mathit{X}$. ${\left(\mathit{U}{\mathit{U}}^T\right)}_{ij}={\displaystyle {\sum }_{l=1}^n{u}_{il}{u}_{jl}}$ represents the inner product between the $i$-th and $j$-th rows of the basis matrix $\mathit{U}$. To maintain consistency between feature relationships in the low-dimensional space and original data, we construct the feature preservation constraint term:

$$\underset{\mathit{U}\ge 0}{\min }{‖\mathit{X}{\mathit{X}}^T-\beta \mathit{U}{\mathit{U}}^T‖}_F^2$$

(19)

where $\beta$ is a non-negative balance parameter.

Equation (19) defines the reconstruction error based on the Frobenius norm. Therefore, when seeking to minimize the objective function value, Equation (19) is susceptible to interference from noise and outliers. To address this, this paper employs the $l_{2,1}$-norm to further optimize Equation (19), thereby enhancing its robustness. The specific expression is as follows:

$$\underset{\mathit{U}\ge 0}{\min }{‖\mathit{X}{\mathit{X}}^T-\beta \mathit{U}{\mathit{U}}^T‖}_{2,1}$$

(20)

3.4. Sparsity of Coefficient Matrix

In the clustering process, the coefficient matrix $\mathit{V}$ plays the role of a cluster indicator matrix. Therefore, the state of the coefficient matrix $\mathit{V}$ affects the accuracy of clustering results. Imposing sparse constraints on the coefficient matrix $\mathit{V}$ can make the representation of each sample in the low-dimensional space significantly correlated with only a few basis vectors, thereby weakening the interference of irrelevant information on clustering results. Common sparsity methods include: $l_0$-norm, $l_{1/2}$-norm, $l_1$-norm, $l_{2,1}$-norm, $l_{\log }$-(pseudo) norm, etc. A detailed introduction is shown in

Table 1. Introduction to five common sparsity methods.

No.	Method	Introduction to Sparsity Method
1	$l_0$-norm [34]	${‖\mathit{V}‖}_0$ denotes the number of non-zero elements in $\mathit{V}$.
2	$l_{1/2}$-norm [35]	${‖\mathit{V}‖}_{1/2}={\displaystyle \sum _{i=1}^r{\displaystyle \sum _{j=1}^n{\left|{\mathit{v}}_{ij}\right|}^{1/2}}}$
3	$l_1$-norm [22]	${‖\mathit{V}‖}_1={\displaystyle \sum _{i=1}^r{\displaystyle \sum _{j=1}^n\left|{\mathit{v}}_{ij}\right|}}$
4	$l_{2,1}$-norm [36]	${‖\mathit{V}‖}_{2,1}={\displaystyle \sum _{j=1}^n\sqrt{{\displaystyle \sum _{i=1}^r{\mathit{v}}_{ij}^2}}}$
5	$l_{\log }$-(pseudo) norm [24]	${‖\mathit{V}‖}_{\log }={\displaystyle \sum _{i=1}^r{\displaystyle \sum _{j=1}^n\log \left(1+\left|{\mathit{v}}_{ij}\right|\right)}}$

, where ${\mathit{v}}_{ij}$ represents the element in the $i$-th row and $j$-th column of the coefficient matrix $\mathit{V}\in {ℝ}_{+}^{r\times n}$.

The five common sparsity methods in Table 1 primarily impose element-wise regularization on the coefficient matrix $\mathit{V}$ but overlook the correlations within the internal structure of $\mathit{V}$. This element-level separate optimization struggles to capture potential high-order collaborative relationships in the sample space. Therefore, this paper proposes a sparse constraint based on inner product penalty.

For the coefficient matrix $\mathit{V}=\left({\mathit{v}}_1,{\mathit{v}}_2,\cdots ,{\mathit{v}}_n\right)\in {ℝ}_{+}^{r\times n}$, we have

$${\mathit{V}}^T\mathit{V}=\left[\begin{array}{cccc}〈{\mathit{v}}_1,{\mathit{v}}_1〉& 〈{\mathit{v}}_1,{\mathit{v}}_2〉& \cdots & 〈{\mathit{v}}_1,{\mathit{v}}_n〉\\ 〈{\mathit{v}}_2,{\mathit{v}}_1〉& 〈{\mathit{v}}_2,{\mathit{v}}_2〉& \cdots & 〈{\mathit{v}}_2,{\mathit{v}}_n〉\\ ⋮& ⋮& \ddots & ⋮\\ 〈{\mathit{v}}_n,{\mathit{v}}_1〉& 〈{\mathit{v}}_n,{\mathit{v}}_2〉& \cdots & 〈{\mathit{v}}_n,{\mathit{v}}_n〉\end{array}\right]$$

(21)

In Equation (21), the diagonal elements of ${\mathit{V}}^T\mathit{V}$ reflect autocorrelation, while the off-diagonal elements reflect correlations between different vectors. Here, to avoid interference from autocorrelation, we remove the diagonal elements based on Equation (21) and only retain the off-diagonal elements for sparsification. Thus, we can further obtain the following:

$${\displaystyle \sum _{i=1}^n{\displaystyle \sum _{j=1}^n〈{\mathit{v}}_i,{\mathit{v}}_j〉}}-{\displaystyle \sum _{i=1}^n〈{\mathit{v}}_i,{\mathit{v}}_i〉}={\displaystyle \sum _{i=1}^n{\displaystyle \sum _{\begin{array}{l}j=1\\ j\ne i\end{array}}^n〈{\mathit{v}}_i,{\mathit{v}}_j〉}}=\mathrm{Tr}\left({\mathit{V}}^T\mathit{V}{\mathbf{1}}_n\right)-\mathrm{Tr}\left({\mathit{V}}^T\mathit{V}\right)$$

(22)

where ${\mathbf{1}}_n$ denotes an $n\times n$ matrix with all-one element.

Therefore, the sparse constraint term for the coefficient matrix $\mathit{V}$ is as follows:

$$\underset{\mathit{V}\ge 0}{\min }\left(\mathrm{Tr}\left({\mathit{V}}^T\mathit{V}{\mathbf{1}}_n\right)-\mathrm{Tr}\left({\mathit{V}}^T\mathit{V}\right)\right)$$

(23)

3.5. Objective Function

From Equations (10), (18), (20) and (23), the objective function of ASNMF-SRP is derived as follows:

$$\begin{array}{l}\underset{\mathit{U},\mathit{V}\ge 0}{\min }{‖\mathit{X}-\mathit{U}\mathit{V}‖}_F^2+{‖\mathit{V}-{\mathit{U}}^T\mathit{X}‖}_F^2+\alpha \mathrm{Tr}\left(\mathit{V}\mathit{L}{\mathit{V}}^T\right)+{‖{\mathit{X}\mathit{X}}^T-\beta {\mathit{U}\mathit{U}}^T‖}_{2,1}\\ \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em} +\lambda \left(\mathrm{Tr}\left({\mathit{V}}^T\mathit{V}{\mathbf{1}}_n\right)-\mathrm{Tr}\left({\mathit{V}}^T\mathit{V}\right)\right)\end{array}$$

(24)

where $\alpha$, $\beta$, and $\lambda$ are all non-negative regularization parameters. Figure 4 is a schematic diagram of the ASNMF-SRP algorithm.

In the objective function Equation (24), each term plays an important role. The first term (${‖\mathit{X}-\mathit{U}\mathit{V}‖}_F^2$) aims to focus on the reconstruction error of matrix factorization. The second term (${‖\mathit{V}-{\mathit{U}}^T\mathit{X}‖}_F^2$) represents an explicit constraint on the coefficient matrix, thereby improving the factorization stability of the coefficient matrix. The third term ($\mathrm{Tr}\left({\mathit{V}\mathit{L}\mathit{V}}^T\right)$) encodes higher-order topological information between samples by constructing the optimal Laplacian matrix. The fourth term (${‖{\mathit{X}\mathit{X}}^T-\beta {\mathit{U}\mathit{U}}^T‖}_{2,1}$) ensures that the feature relationship structure of the original high-dimensional data can be preserved in the low-dimensional space after dimensionality reduction. The fifth term ($\mathrm{Tr}\left({\mathit{V}}^T\mathit{V}{\mathbf{1}}_n\right)-\mathrm{Tr}\left({\mathit{V}}^T\mathit{V}\right)$) imposes a structured sparse constraint on the coefficient matrix by introducing a sparse regularization term in the form of inner product.

3.6. Optimization Algorithm

ASNMF-SRP employs multiplicative update rules to iteratively optimize the basis matrix $\mathit{U}$ and coefficient matrix $\mathit{V}$. The augmented Lagrangian function of the objective function Equation (24) is as follows:

$$\begin{array}{l}\mathcal{P}\left(\mathit{U},\mathit{V}\right)={‖\mathit{X}-\mathit{U}\mathit{V}‖}_F^2+{‖\mathit{V}-{\mathit{U}}^T\mathit{X}‖}_F^2+\alpha \mathrm{Tr}\left({\mathit{V}\mathit{L}\mathit{V}}^T\right)+{‖{\mathit{X}\mathit{X}}^T-\beta {\mathit{U}\mathit{U}}^T‖}_{2,1}\\ \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}+\lambda \left(\mathrm{Tr}\left({\mathit{V}}^T\mathit{V}{\mathbf{1}}_n\right)-\mathrm{Tr}\left({\mathit{V}}^T\mathit{V}\right)\right)-\mathrm{Tr}\left({\mathbf{\Lambda }}_1{\mathit{U}}^T\right)-\mathrm{Tr}\left({\mathbf{\Lambda }}_2{\mathit{V}}^T\right)\end{array}$$

(25)

where ${\mathbf{\Lambda }}_{\mathbf{1}}\in {ℝ}_{+}^{m\times r}$ and ${\mathbf{\Lambda }}_{\mathbf{2}}\in {ℝ}_{+}^{r\times n}$ denote the Lagrange multipliers for $\mathit{U}$ and $\mathit{V}$, respectively.

Expanding Equation (25) and omitting terms independent of $\mathit{U}$ and $\mathit{V}$, we obtain the following:

$$\begin{array}{l}\mathcal{L}\left(\mathit{U},\mathit{V}\right)=\mathrm{Tr}\left({\mathit{V}}^T{\mathit{U}}^T\mathit{U}\mathit{V}-2{\mathit{V}}^T{\mathit{U}}^T\mathit{X}\right)+\mathrm{Tr}\left({\mathit{V}}^T\mathit{V}-2{\mathit{V}}^T{\mathit{U}}^T\mathit{X}+{\mathit{X}}^T\mathit{U}{\mathit{U}}^T\mathit{X}\right)\\ \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}+\alpha \mathrm{Tr}\left(\mathit{V}\mathit{L}{\mathit{V}}^T\right)+\mathrm{Tr}\left({\beta }^2\mathit{U}{\mathit{U}}^T{\mathit{G}}_2\mathit{U}{\mathit{U}}^T-2\beta \mathit{U}{\mathit{U}}^T{\mathit{G}}_2\mathit{X}{\mathit{X}}^T\right)\\ \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}+\lambda \left(\mathrm{Tr}\left({\mathit{V}}^T\mathit{V}{\mathbf{1}}_n\right)-\mathrm{Tr}\left({\mathit{V}}^T\mathit{V}\right)\right)-\mathrm{Tr}\left({\mathbf{\Lambda }}_1{\mathit{U}}^T\right)-\mathrm{Tr}\left({\mathbf{\Lambda }}_2{\mathit{V}}^T\right)\end{array}$$

(26)

where ${\mathit{G}}_2$ is a diagonal matrix whose diagonal elements are as follows:

$${\left({\mathit{G}}_2\right)}_{jj}=1/{‖{\left({\mathit{X}\mathit{X}}^T-\beta {\mathit{U}\mathit{U}}^T\right)}_i‖}_2$$

(27)

Taking partial derivatives of $\mathcal{L}\left(\mathit{U},\mathit{V}\right)$ with respect to $\mathit{U}$ and $\mathit{V}$, respectively, gives the following:

$$\begin{array}{l}{\displaystyle \frac{\mathsf{\partial }\mathcal{L}}{\mathsf{\partial }\mathit{U}}}=2{\mathit{U}\mathit{V}\mathit{V}}^T-4{\mathit{X}\mathit{V}}^T+2{\mathit{X}\mathit{X}}^T\mathit{U}+2{\beta }^2{\mathit{U}\mathit{U}}^T{\mathit{G}}_2\mathit{U}+2{\beta }^2{\mathit{G}}_2{\mathit{U}\mathit{U}}^T\mathit{U}\\ \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}-2\beta {\mathit{X}\mathit{X}}^T{\mathit{G}}_2\mathit{U}-2\beta {\mathit{G}}_2{\mathit{X}\mathit{X}}^T\mathit{U}-{\mathbf{\Lambda }}_{\mathbf{1}}\end{array}$$

(28)

$${\displaystyle \frac{\mathsf{\partial }\mathcal{L}}{\mathsf{\partial }\mathit{V}}}=2{\mathit{U}}^T\mathit{U}\mathit{V}-4{\mathit{U}}^T\mathit{X}+2\mathit{V}+2\alpha \mathit{V}\mathit{L}+2\lambda \mathit{V}{\mathbf{1}}_n-2\lambda \mathit{V}-{\mathbf{\Lambda }}_2$$

(29)

To ensure the non-negativity of the coefficient matrix $\mathit{V}$, the optimal Laplacian matrix $\mathit{L}$ in Equation (29) is factorized as $\mathit{L}={\mathit{L}}^{+}-{\mathit{L}}^{-}$, where ${\mathit{L}}^{+}=(|\mathit{L}|+\mathit{L})/2$ and ${\mathit{L}}^{-}=(|\mathit{L}|-\mathit{L})/2$.

Equation (29) can be rewritten as follows:

$${\displaystyle \frac{\mathsf{\partial }\mathcal{L}}{\mathsf{\partial }\mathit{V}}}=2{\mathit{U}}^T\mathit{U}\mathit{V}-4{\mathit{U}}^T\mathit{X}+2\mathit{V}+2\alpha \mathit{V}{\mathit{L}}^{+}-2\alpha \mathit{V}{\mathit{L}}^{-}+2\lambda \mathit{V}{1}_n-2\lambda \mathit{V}-{\mathbf{\Lambda }}_2$$

(30)

According to the KKT conditions ${\left({\mathbf{\Lambda }}_1\right)}_{ij}{\mathit{U}}_{ij}=0$ and ${\left({\mathbf{\Lambda }}_2\right)}_{ij}{\mathit{V}}_{ij}=0$, we derive the following:

$$\begin{array}{l}(2\mathit{U}\mathit{V}{\mathit{V}}^T-4\mathit{X}{\mathit{V}}^T+2\mathit{X}{\mathit{X}}^T\mathit{U}+2{\beta }^2\mathit{U}{\mathit{U}}^T{\mathit{G}}_2\mathit{U}+2{\beta }^2{\mathit{G}}_2\mathit{U}{\mathit{U}}^T\mathit{U}\\ \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}-2\beta \mathit{X}{\mathit{X}}^T{\mathit{G}}_2\mathit{U}-2\beta {\mathit{G}}_2\mathit{X}{\mathit{X}}^T\mathit{U}-{\mathbf{\Lambda }}_{\mathbf{1}}{)}_{ij}{\mathit{U}}_{ij}=0\end{array}$$

(31)

$${\left(2{\mathit{U}}^T\mathit{U}\mathit{V}-4{\mathit{U}}^T\mathit{X}+2\mathit{V}+2\alpha \mathit{V}{\mathit{L}}^{+}-2\alpha \mathit{V}{\mathit{L}}^{-}+2\lambda \mathit{V}{\mathbf{1}}_n-2\lambda \mathit{V}-{\mathbf{\Lambda }}_2\right)}_{ij}{\mathit{V}}_{ij}=0$$

(32)

Therefore, we obtain the following:

$$\begin{array}{l}(2\mathit{U}\mathit{V}{\mathit{V}}^T-4\mathit{X}{\mathit{V}}^T+2\mathit{X}{\mathit{X}}^T\mathit{U}+2{\beta }^2\mathit{U}{\mathit{U}}^T{\mathit{G}}_2\mathit{U}+2{\beta }^2{\mathit{G}}_2\mathit{U}{\mathit{U}}^T\mathit{U}\\ \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}-2\beta \mathit{X}{\mathit{X}}^T{\mathit{G}}_2\mathit{U}-2\beta {\mathit{G}}_2\mathit{X}{\mathit{X}}^T\mathit{U}-{\mathbf{\Lambda }}_{\mathbf{1}}{)}_{ij}{\mathit{U}}_{ij}^2=0\end{array}$$

(33)

$${\left(2{\mathit{U}}^T\mathit{U}\mathit{V}-4{\mathit{U}}^T\mathit{X}+2\mathit{V}+2\alpha \mathit{V}{\mathit{L}}^{+}-2\alpha \mathit{V}{\mathit{L}}^{-}+2\lambda \mathit{V}{\mathbf{1}}_n-2\lambda \mathit{V}-{\mathbf{\Lambda }}_2\right)}_{ij}{\mathit{V}}_{ij}^2=0$$

(34)

From Equations (33) and (34), the update formulas for $\mathit{U}$ and $\mathit{V}$ are as follows:

$${\mathit{U}}_{ij}^{t+1}\leftarrow {\mathit{U}}_{ij}^t\sqrt{{\displaystyle \frac{{\left(2{\mathit{X}\mathit{V}}^T+\beta {\mathit{X}\mathit{X}}^T{\mathit{G}}_2\mathit{U}+\beta {\mathit{G}}_2{\mathit{X}\mathit{X}}^T\mathit{U}\right)}_{ij}}{{\left({\mathit{U}\mathit{V}\mathit{V}}^T+{\mathit{X}\mathit{X}}^T\mathit{U}+{\beta }^2{\mathit{U}\mathit{U}}^T{\mathit{G}}_2\mathit{U}+{\beta }^2{\mathit{G}}_2{\mathit{U}\mathit{U}}^T\mathit{U}\right)}_{ij}}}}$$

(35)

$${\mathit{V}}_{ij}^{t+1}\leftarrow {\mathit{V}}_{ij}^t\sqrt{{\displaystyle \frac{{\left(2{\mathit{U}}^T\mathit{X}+\alpha \mathit{V}{\mathit{L}}^{-}+\lambda \mathit{V}\right)}_{ij}}{{\left({\mathit{U}}^T\mathit{U}\mathit{V}+\mathit{V}+\alpha \mathit{V}{\mathit{L}}^{+}+\lambda \mathit{V}{\mathbf{1}}_n\right)}_{ij}}}}$$

(36)

3.7. Convergence Analysis

In this section, the convergence of $\mathit{U}$ and $\mathit{V}$ under the update rules in Equations (35) and (36) in the ASNMF-SRP algorithm is proven using the auxiliary function method.

Firstly, we prove that the coefficient matrix $\mathit{V}$ is non-increasing under the update rule in Equation (36). According to the objective function Equation (24), we can derive a functional $F_{ij}\left({\mathit{V}}_{ij}\right)$ regarding the coefficient matrix $\mathit{V}$ as follows:

$$F_{ij}({\mathit{V}}_{ij})=\mathrm{Tr}({\mathit{V}}^T{\mathit{U}}^T\mathit{U}\mathit{V}-4{\mathit{V}}^T{\mathit{U}}^T\mathit{X}+{\mathit{V}}^T\mathit{V}+\alpha \mathit{V}{\mathit{L}}^{+}{\mathit{V}}^T-\alpha \mathit{V}{\mathit{L}}^{-}{\mathit{V}}^T+\lambda {\mathit{V}}^T\mathit{V}{\mathbf{1}}_n-\lambda {\mathit{V}}^T\mathit{V})$$

(37)

Proposition 1.

The constructed function $\mathit{G}\left({\mathit{V}}_{ij},{\mathit{V}}_{ij}^t\right)$ is an auxiliary function of $F_{ij}\left({\mathit{V}}_{ij}\right)$.

$$\begin{array}{l}\mathit{G}\left({\mathit{V}}_{ij},{\mathit{V}}_{ij}^t\right)={\displaystyle \sum _{ij}{\left({\mathit{U}}^T\mathit{U}{\mathit{V}}^t\right)}_{ij}\frac{{\mathit{V}}_{ij}^2}{{\mathit{V}}_{ij}^t}}-4{\displaystyle \sum _{ij}{\left({\mathit{U}}^T\mathit{X}\right)}_{ij}{\mathit{V}}_{ij}^t\left(1+\log \frac{{\mathit{V}}_{ij}}{{\mathit{V}}_{ij}^t}\right)}\\ \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}+{\displaystyle \sum _{ij}{\mathit{V}}_{ij}^2}+\alpha {\displaystyle \sum _{ij}{\left({\mathit{V}}^t{\mathit{L}}^{+}\right)}_{ij}\frac{{\mathit{V}}_{ij}^2}{{\mathit{V}}_{ij}^t}}-\alpha {\displaystyle \sum _{ijl}\left({\mathit{L}}_{jl}^{-}{\mathit{V}}_{ij}^t{\mathit{V}}_{il}^t\right)\left(1+\log \frac{{\mathit{V}}_{jl}{\mathit{V}}_{il}}{{\mathit{V}}_{jl}^t{\mathit{V}}_{il}^t}\right)}\\ \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}+\lambda {\displaystyle \sum _{ij}{\left({\mathit{V}}^t{1}_n\right)}_{ij}}{\displaystyle \frac{{\mathit{V}}_{ij}^2}{{\mathit{V}}_{ij}^t}}-\lambda {\displaystyle \sum _{ij}{\mathit{V}}_{ij}^t{\mathit{V}}_{ij}^t}\left(1+\log {\displaystyle \frac{{\mathit{V}}_{ij}{\mathit{V}}_{ij}}{{\mathit{V}}_{ij}^t{\mathit{V}}_{ij}^t}}\right)\end{array}$$

(38)

Proof. 

When ${\mathit{V}}_{ij}={\mathit{V}}_{ij}^t$, we have $\mathit{G}\left({\mathit{V}}_{ij},{\mathit{V}}_{ij}\right)=F_{ij}\left({\mathit{V}}_{ij}\right)$. The following proves the case $\mathit{G}\left({\mathit{V}}_{ij},{\mathit{V}}_{ij}^t\right)\ge F_{ij}\left({\mathit{V}}_{ij}\right)$.

When $\mathit{A}$ and $\mathit{B}$ are symmetric matrices, we have the following:

$$\begin{array}{l}{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{j=1}^m\frac{{\left(\mathit{A}{\mathit{S}}^t\mathit{B}\right)}_{ij}{\mathit{S}}_{ij}^2}{{\mathit{S}}_{ij}^t}\ge \mathrm{Tr}\left({\mathit{S}}^T\mathit{A}\mathit{S}\mathit{B}\right)}},\\ \forall \hspace{0.33em}\mathit{A}\in {ℝ}_{+}^{n\times n},\hspace{0.33em}\mathit{B}\in {ℝ}_{+}^{m\times m},\hspace{0.33em}{\mathit{S}}^t\in {ℝ}_{+}^{n\times m},\hspace{0.33em}\mathit{S}\in {ℝ}_{+}^{n\times m}\end{array}$$

(39)

From Equation (39), we obtain the following:

$$\mathrm{Tr}\left({\mathit{V}}^T{\mathit{U}}^T\mathit{U}\mathit{V}\right)\le {\displaystyle \sum _{ij}{\left({\mathit{U}}^T\mathit{U}{\mathit{V}}^t\right)}_{ij}}{\displaystyle \frac{{\mathit{V}}_{ij}^2}{{\mathit{V}}_{ij}^t}}$$

(40)

$$\mathrm{Tr}\left({\mathit{V}}^T\mathit{V}\right)\le {\displaystyle \sum _{ij}{\mathit{V}}_{ij}^2}$$

(41)

$$\alpha \mathrm{Tr}\left(\mathit{V}{\mathit{L}}^{+}{\mathit{V}}^T\right)\le \alpha {\displaystyle \sum _{ij}{\left({\mathit{V}}^t{\mathit{L}}^{+}\right)}_{ij}\frac{{\mathit{V}}_{ij}^2}{{\mathit{V}}_{ij}^t}}$$

(42)

$$\lambda \mathrm{Tr}\left({\mathit{V}}^T\mathit{V}{\mathbf{1}}_n\right)\le \lambda {\displaystyle \sum _{ij}{\left({\mathit{V}}^t{\mathbf{1}}_n\right)}_{ij}}{\displaystyle \frac{{\mathit{V}}_{ij}^2}{{\mathit{V}}_{ij}^t}}$$

(43)

From $\mathit{x}>1+\log \mathit{x}\left(\mathit{x}>0\right)$, we obtain the following:

$$-4\mathrm{Tr}\left({\mathit{V}}^T{\mathit{U}}^T\mathit{X}\right) =-4\mathrm{Tr}\left({\mathit{X}}^T\mathit{U}\mathit{V}\right)\le -4{\displaystyle \sum _{ij}{\left({\mathit{U}}^T\mathit{X}\right)}_{ij}{\mathit{V}}_{ij}^t}\left(1+\log {\displaystyle \frac{{\mathit{V}}_{ij}}{{\mathit{V}}_{ij}^t}}\right)$$

(44)

$$-\alpha \mathrm{Tr}\left(\mathit{V}{\mathit{L}}^{-}{\mathit{V}}^T\right)\le -\alpha {\displaystyle \sum _{ijl}\left({\mathit{L}}_{jl}^{-}{\mathit{V}}_{ij}^t{\mathit{V}}_{il}^t\right)\left(1+\log \frac{{\mathit{V}}_{jl}{\mathit{V}}_{il}}{{\mathit{V}}_{jl}^t{\mathit{V}}_{il}^t}\right)}$$

(45)

$$-\lambda \mathrm{Tr}\left({\mathit{V}}^T\mathit{V}\right)\le -\lambda {\displaystyle \sum _{ij}{\mathit{V}}_{ij}^t{\mathit{V}}_{ij}^t}\left(1+\log {\displaystyle \frac{{\mathit{V}}_{ij}{\mathit{V}}_{ij}}{{\mathit{V}}_{ij}^t{\mathit{V}}_{ij}^t}}\right)$$

(46)

From Equations (40)–(46), it follows that $\mathit{G}\left({\mathit{V}}_{ij},{\mathit{V}}_{ij}^t\right)\ge F_{ij}\left({\mathit{V}}_{ij}\right)$. Therefore, $\mathit{G}\left({\mathit{V}}_{ij},{\mathit{V}}_{ij}^t\right)$ is an auxiliary function of $F_{ij}\left({\mathit{V}}_{ij}\right)$, and Proposition 1 is proven.

Let ${\displaystyle \frac{\mathsf{\partial }\mathit{G}\left(\mathit{V},{\mathit{V}}^t\right)}{\mathsf{\partial }{\mathit{V}}_{ij}}}=0$, we obtain the following:

$${\mathit{V}}_{ij}^{t+1}\leftarrow {\mathit{V}}_{ij}^t\sqrt{{\displaystyle \frac{{\left(2{\mathit{U}}^T\mathit{X}+\alpha \mathit{V}{\mathit{L}}^{-}+\lambda \mathit{V}\right)}_{ij}}{{\left({\mathit{U}}^T\mathit{U}\mathit{V}+\mathit{V}+\alpha \mathit{V}{\mathit{L}}^{+}+\lambda \mathit{V}{\mathbf{1}}_n\right)}_{ij}}}}$$

(47)

Thus, the coefficient matrix $\mathit{V}$ in the ASNMF-SRP algorithm is non-increasing under the update rule in Equation (36). Similarly, it can be proven that the basis matrix $\mathit{U}$ is non-increasing under the update rule in Equation (35). Therefore, $\mathit{U}$ and $\mathit{V}$ converge under the update rules in Equations (35) and (36). The specific implementation of the ASNMF-SRP algorithm is described in Algorithm 1.

Algorithm 1 ASNMF-SRP

Input: Initial matrix $\mathit{X}=\left({\mathit{x}}_1,{\mathit{x}}_2,\cdots ,{\mathit{x}}_n\right)\in {ℝ}_{+}^{m\times n}$, number of classes $r$, neighborhood parameter $k$, regularization parameters $\alpha$, $\beta$ and $\lambda$, balance parameters $w_1$ and $w_2$ parameter $d$, threshold $\epsilon$, maximum iterations $t$. Output: Basis matrix $\mathit{U}$ and coefficient matrix $\mathit{V}$. 1. Initialization: $t=0$, Randomly generate basis matrix $\mathit{U}\in {ℝ}^{\mathrm{m}\times r}$ and coefficient matrix $\mathit{V}\in {ℝ}^{r\times n}$; 2. Obtain optimal Laplacian matrix $\mathit{L}$ according to Equations (11)–(17); 3. For $t=1,2,3,\cdots ,\mathrm{maxIter}$ 4. ${\mathit{U}}_{ij}^{t+1}\leftarrow {\mathit{U}}_{ij}^t\sqrt{{\displaystyle \frac{{\left(2{\mathit{X}\mathit{V}}^T+\beta {\mathit{X}\mathit{X}}^T{\mathit{G}}_2\mathit{U}+\beta {\mathit{G}}_2{\mathit{X}\mathit{X}}^T\mathit{U}\right)}_{ij}}{{\left({\mathit{U}\mathit{V}\mathit{V}}^T+{\mathit{X}\mathit{X}}^T\mathit{U}+{\beta }^2{\mathit{U}\mathit{U}}^T{\mathit{G}}_2\mathit{U}+{\beta }^2{\mathit{G}}_2{\mathit{U}\mathit{U}}^T\mathit{U}\right)}_{ij}}}}$; 5. ${\mathit{V}}_{ij}^{t+1}\leftarrow {\mathit{V}}_{ij}^t\sqrt{{\displaystyle \frac{{\left(2{\mathit{U}}^T\mathit{X}+\alpha \mathit{V}{\mathit{L}}^{-}+\lambda \mathit{V}\right)}_{ij}}{{\left({\mathit{U}}^T\mathit{U}\mathit{V}+\mathit{V}+\alpha \mathit{V}{\mathit{L}}^{+}+\lambda \mathit{V}{\mathbf{1}}_n\right)}_{ij}}}}$; 6. if ${‖{\mathit{U}}^t-{\mathit{U}}^{t-1}‖}_{\infty }\le \epsilon$ and ${‖{\mathit{V}}^t-{\mathit{V}}^{t-1}‖}_{\infty }\le \epsilon$ Break and return ($\mathit{U}$,$\mathit{V}$); 7. End if 8. End for

□

3.8. Time Complexity Analysis

When inputting a data matrix $\mathit{X}\in {ℝ}^{m\times n}$, where $m$ is the feature dimension of samples and $n$ is the number of samples, assume the number of classes for this dataset is $r$, and $r\ll \min \left\{m,n\right\}$. After $t$ iterations, the time complexity of ASNMF-SRP is as follows:

(1) The autoencoder-like NMF part: complexities for updating $\mathit{U}$ and $\mathit{V}$ are $O\left(t{m}^{2}r+tm{r}^2\right)$ and $O\left(t{m}^{2}r+tm{r}^2\right)$;

(2) The higher-order graph regularization part: complexity is $O\left(m{n}^2+m{e}^d\right)$, where $e$ denotes the average degree of nodes;

(3) The feature preservation part: complexity is $O\left(t{m}^{2}r+t\left(m^2+m\right)\right)$;

(4) The sparse constraint term: complexity is $O\left(t{n}^{2}r\right)$.

Therefore, the overall time complexity of ASNMF-SRP is $O\left(t{m}^{2}r+t\left(m^2+m\right)+t{n}^{2}r+m{n}^2+m{e}^d\right)$.

4. Experiments

In this section, clustering experiments will be conducted on 8 public datasets to evaluate the clustering performance of the ASNMF-SRP algorithm. The experiments are implemented in Python 3.11 with the computer environment being Intel(R) Core (TM) i5-1135G7 @ 2.40 GHz 2.42 GHz, 16 GB RAM, and Windows 11 64-bit operating system.

4.1. Dataset

To verify the effectiveness of the algorithm, we selected 8 public datasets for comparative experiments. MSRA25 [37] dataset contains 1799 face images from 12 individuals. Semeion, Krvs, PenDigits, and Vehicle datasets are from the UCI Machine Learning Repository (https://archive.ics.uci.edu (accessed on 23 March 2025)). Hitech dataset is from the CLUTO toolkit (https://conservancy.umn.edu/items/4fbef165-f964-41ed-a239-86a8f931ffbe (accessed on 26 March 2025)). COIL20 (http://www.cs.columbia.edu/CAVE/software/softlib/coil-20.php (accessed on 15 December 2024)) and COIL100 (http://www.kaggle.com/jessicali9530/coil100/downloads/coil100.zip/2 (accessed on 15 December 2024)) datasets record images of 20 and 100 objects, respectively. The basic information of these 8 datasets is shown in

Table 2. Introduction of the 8 datasets.

NO.	Dataset	Samples ($\mathit{n}$)	Features ($\mathit{m}$)	Classes ($\mathit{r}$)	Data Type	Image Size
1	MSRA25	1799	256	12	Face dataset	16 × 16
2	Semeion	1593	256	10	Digit images	16 × 16
3	COIL20	1440	1024	20	Object images	32 × 32
4	COIL100	7200	1024	100	Object images	32 × 32
5	Krvs	3196	36	2	Network detection	—
6	Hitech	2301	2216	6	Technology news	—
7	PenDigits	3498	16	10	Handwritten digits	—
8	Vehicle	846	18	4	Vehicle contours	—

.

4.2. Clustering Performance Evaluation Metrics

To compare the clustering performance of various algorithms, this paper selects four commonly used clustering performance evaluation metrics. Among them, larger values for these four clustering evaluation metrics indicate better clustering performance of the corresponding algorithm.

4.2.1. Clustering Accuracy (ACC)

$$\mathrm{ACC}={\displaystyle \frac{{\displaystyle \sum _{i=1}^n\delta \left(map\left(s_i\right),r_i\right)}}{n}}$$

(48)

where $n$ is the total number of samples in the input data, $r_i$ is the true class label of the data, $s_i$ represents the result after clustering by the algorithm, and $map\left(\cdot \right)$ denotes the mapping function. The expression $\delta \left(map\left(s_i\right),r_i\right)$ in Equation (48) is as follows:

$$\delta \left(map\left(s_i\right),r_i\right)=\left\{\begin{array}{c}1,\hspace{1em}map\left(s_i\right)=r_i\hfill \\ 0,\hspace{1em}map\left(s_i\right)\ne r_i\end{array}\right.$$

(49)

From Equations (48) and (49), ACC is an evaluation metric used to measure the consistency between clustering results and true labels, with a value range of $\left[0,1\right]$.

4.2.2. Adjusted Rand Index (ARI)

$$\mathrm{ARI}={\displaystyle \frac{{\displaystyle {\sum }_{ij}\left(\begin{array}{c}{n}_{ij}\\ 2\end{array}\right)-\left[{\displaystyle {\sum }_i\left(\begin{array}{c}{\mathit{x}}_i\\ 2\end{array}\right){\displaystyle {\sum }_j\left(\begin{array}{c}{y}_i\\ 2\end{array}\right)}}\right]/\left(\begin{array}{c}n\\ 2\end{array}\right)}}{0.5\left[{\displaystyle {\sum }_i\left(\begin{array}{c}{\mathit{x}}_i\\ 2\end{array}\right)+{\displaystyle {\sum }_j\left(\begin{array}{c}{y}_i\\ 2\end{array}\right)}}\right]-\left[{\displaystyle {\sum }_i\left(\begin{array}{c}{\mathit{x}}_i\\ 2\end{array}\right){\displaystyle {\sum }_j\left(\begin{array}{c}{y}_i\\ 2\end{array}\right)}}\right]/\left(\begin{array}{c}n\\ 2\end{array}\right)}}$$

(50)

where ${\mathit{x}}_i$ represents the total number of samples in the $i$-th cluster after clustering, $y_i$ represents the total number of samples in the true $j$-th class, and $n_{ij}$ denotes the total number of samples common to both the cluster ${\mathit{x}}_i$ and the true class $y_j$. From Equation (50), the value range of ARI is $\left[-1,1\right]$.

4.2.3. Normalized Mutual Information (NMI)

$$\mathrm{NMI}\left(N,N^{*}\right)={\displaystyle \frac{\mathrm{MI}\left(N,N^{*}\right)}{\max \left(H\left(N\right),H\left(N^{*}\right)\right)}}$$

(51)

where $N^{*}$ represents the data labels obtained through clustering, $N$ represents the actual data labels, and $H\left(\cdot \right)$ is defined as the entropy function.

The mutual information $\mathrm{MI}\left(\cdot \right)$ expression is as follows:

$$\mathrm{MI}\left(N,N^{*}\right)={\displaystyle \sum _{n_i\in N,n_j^{*}\in N^{*}}p\left(n_i,n_j^{*}\right)}\log {\displaystyle \frac{p\left(n_i,n_j^{*}\right)}{p\left(n_i\right)p\left(n_j^{*}\right)}}$$

(52)

where $p\left(n_i\right)$ and $p\left(n_j^{*}\right)$ represent the marginal probabilities of the true class and clustering results, respectively, and $p\left(n_i,n_j^{*}\right)$ is the joint probability of the true class $n_i$ and cluster $n_j^{*}$. From Equations (51) and (52), the value range of NMI is $\left[0,1\right]$.

4.2.4. Clustering Purity (PUR)

$$\mathrm{PUR}={\displaystyle \sum _{i=1}^r\frac{{\max }_j\left(n_i^j\right)}{n}}$$

(53)

where $n_i$ represents the total number of samples, $r$ is the number of clusters, and $n_i^j$ represents the number of samples in cluster $i$ that belong to the true class $j$. From Equation (53), the value range of PUR is $\left[0,1\right]$.

4.3. Comparison Algorithms and Parameter Settings

The detailed descriptions of the 9 algorithms participating in the comparative experiments are as follows:

(1) NMF [6] factorizes the original non-negative matrix into the product of two non-negative low-rank submatrices to achieve the goal of representing the whole by parts.

(2) ONMF [9] imposes orthogonality constraints on the factorized submatrices based on NMF.

(3) Hx-NMF [38] combines logarithm with NMF to improve the robustness of the algorithm.

(4) EMMF [39] is an algorithm based on entropy minimization matrix factorization.

(5) GNMF [11] encodes the local geometric structure between data into the objective function to improve the quality of low-dimensional representations.

(6) RMMMF [40] uses the $l_{2,1}$-norm to enhance the algorithm’s robustness.

(7) DRCC [27] jointly optimizes the local geometric structures of samples and features by incorporating them into the objective function.

(8) FR-NMF [28] improves the clustering performance of the algorithm by introducing a feature preservation term.

(9) LS-NMF [24] imposes sparse constraints on the basis matrix and coefficient matrix using the $l_{log}$-(pseudo) norm.

In the experimental process, we perform a grid search over the algorithm’s regularization parameter in the range $\left\{10^{-4},10^{-3},10^{-2},10^{-1},10^0,10^1,10^2,10^3\right\}$.

During the experiments, the parameter values for the higher-order graph regularization parameter $\alpha$, feature relationship preservation regularization parameter $\beta$, and sparse constraint regularization parameter $\lambda$ in the ASNMF-SRP algorithm are shown in

Table 3. Parameter values of ASNMF-SRP on different datasets.

No.	Dataset	Higher-Order Graph Regularization $\mathit{\alpha }$	Feature Relationship Preservation $\mathit{\beta }$	Sparse Constraint $\mathit{\lambda }$
1	MSRA25	1000	1000	0.001
2	Semeion	1	100	0.0001
3	COIL20	1000	1000	0.001
4	COIL100	1000	1000	0.001
5	Krvs	100	10	0.0001
6	Hitech	1000	10	0.0001
7	PenDigits	100	100	0.001
8	Vehicle	1	1	0.001

.

4.4. Results and Analysis

Due to fluctuations in the clustering results of various algorithms, to obtain representative clustering outcomes, each algorithm was executed 20 times on the 8 datasets during the experiments. The mean and standard deviation were calculated as the final comparative clustering results.

Table 4. ACC of 10 algorithms on 8 datasets (mean ± standard deviation).

	Algorithm	NMF	ONMF	Hx-NMF	EMMF	GNMF	RMNMF	DRCC	FR-NMF	LS-NMF	ASNMF-SRP	I-P
Dataset
MSRA25		0.50842	0.49375	0.52026	0.50592	0.53938	0.55550	0.28974	0.51659	0.54019	0.57904	4.24%
		±0.023	±0.026	±0.029	±0.018	±0.032	±0.034	±0.029	±0.022	±0.022	±0.045	--
Semeion		0.52508	0.56959	0.50807	0.51965	0.59209	0.27916	0.61601	0.53540	0.60251	0.68063	10.49%
		±0.042	±0.032	±0.042	±0.040	±0.039	±0.057	±0.037	±0.037	±0.036	±0.050	--
COIL20		0.66406	0.68531	0.65812	0.65142	0.76844	0.24385	0.79017	0.65028	0.77361	0.84174	6.53%
		±0.029	±0.028	±0.030	±0.019	±0.013	±0.094	±0.034	±0.026	±0.014	±0.012	--
COIL100		0.47026	0.49159	0.46877	0.47882	0.48738	0.41235	0.46099	0.47044	0.48306	0.64173	30.54%
		±0.014	±0.011	±0.012	±0.018	±0.014	±0.012	±0.014	±0.015	±0.011	±0.010	--
Krvs		0.51909	0.53742	0.52223	0.52137	0.53082	0.51810	0.55594	0.52552	0.53387	0.56813	2.19%
		±0.003	±0.012	±0.002	±0.003	±0.012	±0.007	±0.004	±0.023	±0.015	±0.003	--
Hitech		0.23385	0.23375	0.23403	0.23544	0.23105	0.26273	0.24087	0.22603	0.23268	0.25367	−3.45%
		±0.002	±0.005	±0.008	±0.004	±0.004	±0.001	±0.004	±0.007	±0.002	±0.002	--
PenDigits		0.66216	0.70729	0.68092	0.66630	0.67973	0.65183	0.73119	0.66791	0.68533	0.80442	10.02%
		±0.036	±0.048	±0.040	±0.038	±0.053	±0.037	±0.043	±0.034	±0.052	±0.032	--
Vehicle		0.38794	0.43777	0.40142	0.39096	0.44397	0.35916	0.41194	0.43570	0.44368	0.45236	1.89%
		±0.019	±0.002	±0.026	±0.021	±0.008	±0.003	±0.017	±0.023	±0.009	±0.002	--

,

Table 5. ARI of 10 algorithms on 8 datasets (mean ± standard deviation).

	Algorithm	NMF	ONMF	Hx-NMF	EMMF	GNMF	RMNMF	DRCC	FR-NMF	LS-NMF	ASNMF-SRP	I-P
Dataset
MSRA25		0.34575	0.32933	0.35734	0.33482	0.40605	0.40058	0.12013	0.35562	0.40596	0.44662	9.99%
		±0.026	±0.027	±0.027	±0.020	±0.037	±0.034	±0.023	±0.019	±0.034	±0.052	--
Semeion		0.31198	0.35051	0.30655	0.31134	0.44280	0.11855	0.41651	0.31943	0.45921	0.48809	6.29%
		±0.033	±0.024	±0.031	±0.030	±0.032	±0.047	±0.026	±0.029	±0.030	±0.034	--
COIL20		0.57989	0.62582	0.57627	0.57069	0.74160	0.16450	0.73372	0.56682	0.74234	0.80244	8.10%
		±0.026	±0.023	±0.034	±0.026	±0.018	±0.093	±0.036	±0.024	±0.016	±0.007	--
COIL100		0.39584	0.44157	0.39549	0.40697	0.42371	0.30329	0.39167	0.39709	0.42067	0.53573	21.32%
		±0.016	±0.014	±0.017	±0.019	±0.010	±0.015	±0.016	±0.012	±0.011	±0.016	--
Krvs		0.00107	0.00579	0.00158	0.00147	0.00393	−0.00038	0.01201	0.00410	0.00505	0.01814	51.04%
		±0.000	±0.004	±0.000	±0.000	±0.003	±0.001	±0.002	±0.006	±0.005	±0.002	--
Hitech		−0.00095	0.00092	0.00021	0.00034	−0.00041	0.00016	0.00252	0.00066	−0.00049	0.00689	173.41%
		±0.001	±0.001	±0.001	±0.001	±0.001	±0.001	±0.002	±0.001	±0.001	±0.000	--
PenDigits		0.52361	0.55428	0.52352	0.52329	0.54137	0.50238	0.57474	0.52889	0.55809	0.68487	19.16%
		±0.029	±0.040	±0.037	±0.022	±0.045	±0.049	±0.037	±0.024	±0.039	±0.035	--
Vehicle		0.08169	0.12462	0.09207	0.08322	0.13103	0.06364	0.09691	0.11321	0.13221	0.12014	−9.13%
		±0.013	±0.003	±0.018	±0.018	±0.009	±0.004	±0.014	±0.012	±0.007	±0.001	--

,

Table 6. NMI of 10 algorithms on 8 datasets (mean ± standard deviation).

	Algorithm	NMF	ONMF	Hx-NMF	EMMF	GNMF	RMNMF	DRCC	FR-NMF	LS-NMF	ASNMF-SRP	I-P
Dataset
MSRA25		0.56935	0.56296	0.57773	0.55467	0.65111	0.60295	0.23745	0.57715	0.64613	0.71512	9.83%
		±0.021	±0.028	±0.023	±0.021	±0.031	±0.029	±0.033	±0.017	±0.031	±0.026	--
Semeion		0.44162	0.48847	0.44312	0.44938	0.60790	0.20171	0.54014	0.44938	0.61489	0.63282	2.92%
		±0.025	±0.020	±0.026	±0.023	±0.020	±0.074	±0.018	±0.022	±0.019	±0.021	--
COIL20		0.76112	0.79591	0.76067	0.75423	0.88538	0.31375	0.89131	0.75546	0.88500	0.91529	2.69%
		±0.015	±0.010	±0.018	±0.017	±0.012	±0.131	±0.011	±0.015	±0.012	±0.006	--
COIL100		0.75258	0.76835	0.75400	0.75646	0.77226	0.70061	0.74641	0.73117	0.76948	0.83835	8.56%
		±0.005	±0.005	±0.006	±0.006	±0.004	±0.009	±0.006	±0.005	±0.004	±0.003
Krvs		0.00060	0.00397	0.00091	0.00094	0.00265	0.00203	0.00818	0.00592	0.00352	0.01250	52.81%
		±0.000	±0.003	±0.000	±0.000	±0.002	±0.003	±0.001	±0.006	±0.003	±0.001	--
Hitech		0.00799	0.00989	0.00854	0.01049	0.00865	0.00558	0.01203	0.00786	0.00799	0.01935	60.85%
		±0.001	±0.002	±0.002	±0.001	±0.002	±0.001	±0.002	±0.002	±0.002	±0.000	--
PenDigits		0.68251	0.69332	0.66576	0.67684	0.69751	0.64294	0.69615	0.68480	0.70953	0.80120	12.92%
		±0.022	±0.019	±0.026	±0.018	±0.027	±0.043	±0.016	±0.021	±0.018	±0.021	--
Vehicle		0.11678	0.19000	0.13648	0.12714	0.19062	0.08408	0.14858	0.16404	0.19181	0.18544	−3.32%
		±0.014	±0.006	±0.019	±0.018	±0.015	±0.005	±0.019	±0.018	±0.013	±0.000	--

and

Table 7. PUR of 10 algorithms on 8 datasets (mean ± standard deviation).

	Algorithm	NMF	ONMF	Hx-NMF	EMMF	GNMF	RMNMF	DRCC	FR-NMF	LS-NMF	ASNMF-SRP	I-P
Dataset
MSRA25		0.52985	0.52176	0.53755	0.52287	0.56587	0.57918	0.30698	0.53849	0.56651	0.61479	6.15%
		±0.021	±0.023	±0.025	±0.020	±0.026	±0.027	±0.030	±0.017	±0.025	±0.035	--
Semeion		0.53763	0.58804	0.53431	0.53807	0.63726	0.28738	0.63625	0.54862	0.64369	0.69739	8.34%
		±0.036	±0.025	±0.030	±0.031	±0.024	±0.061	±0.027	±0.028	±0.026	±0.031	--
COIL20		0.69135	0.71340	0.68628	0.67997	0.80715	0.24903	0.82802	0.67753	0.80892	0.86455	4.41%
		±0.024	±0.023	±0.021	±0.021	±0.017	±0.095	±0.017	±0.023	±0.018	±0.011	--
COIL100		0.52663	0.54705	0.52816	0.53379	0.54654	0.47951	0.51849	0.51637	0.54247	0.69599	27.23%
		±0.013	±0.010	±0.011	±0.013	±0.012	±0.012	±0.012	±0.012	±0.009	±0.006	--
Krvs		0.52245	0.53742	0.52289	0.52261	0.53137	0.52237	0.55594	0.53360	0.53387	0.56813	2.19%
		±0.000	±0.012	±0.001	±0.000	±0.011	±0.000	±0.004	±0.016	±0.015	±0.003	--
Hitech		0.26693	0.27034	0.26758	0.27017	0.26877	0.26380	0.27099	0.26788	0.26606	0.28525	5.26%
		±0.003	±0.003	±0.003	±0.003	±0.003	±0.001	±0.004	±0.005	±0.003	±0.001	--
PenDigits		0.69262	0.72340	0.69447	0.69118	0.70675	0.67226	0.73872	0.69626	0.71256	0.81095	9.78%
		±0.026	±0.031	±0.031	±0.026	±0.035	±0.035	±0.029	±0.024	±0.035	±0.023	--
Vehicle		0.39285	0.43777	0.40573	0.39681	0.44397	0.37145	0.41832	0.43853	0.44368	0.45236	1.89%
		±0.015	±0.002	±0.020	±0.018	±0.008	±0.005	±0.020	±0.022	±0.009	±0.002	--

present the specific values of ACC, ARI, NMI, and PUR for these 10 algorithms on the 8 datasets, respectively. In the tables, bold numbers indicate the optimal clustering results on that dataset, and the I-P (Improvement Percentage) column shows the percentage by which the ASNMF-SRP algorithm improves over the best result among the other 9 algorithms.

By comparing the data in Table 4, Table 5, Table 6 and Table 7, it can be seen that ASNMF-SRP exhibits excellent clustering performance. The specific data analysis is as follows:

(1) On image datasets (MSRA25, Semeion, COIL20, and COIL100), the values of all four clustering evaluation metrics for ASNMF-SRP are higher than those of the other nine algorithms. This indicates that ASNMF-SRP holds certain advantages in handling clustering problems related to images.

(2) On non-image datasets (Krvs, Hitech, PenDigits, and Vehicle), ASNMF-SRP fails to achieve optimal values in ACC on the Hitech dataset and in ARI and NMI on the Vehicle dataset. However, in most cases, ASNMF-SRP demonstrates optimal performance.

(3) FR-NMF incorporates an additional feature relationship preservation term compared to NMF. According to the data in Table 4, Table 5, Table 6 and Table 7, FR-NMF outperforms NMF in clustering performance on some datasets. ASNMF-SRP not only includes feature relationship preservation but also incorporates components such as autoencoder-like NMF, higher-order graph regularization, and sparse constraints. The data in Table 4, Table 5, Table 6 and Table 7 show that ASNMF-SRP achieves higher values across all four clustering evaluation metrics on these eight datasets compared to NMF and FR-NMF. This demonstrates that through the synergistic effects of various regularization terms, ASNMF-SRP not only enhances clustering performance but also improves generalization capability.

This paper employs the t-SNE method to project the low-dimensional representation matrices obtained by the ten algorithms on the MSRA25, COIL20, and Hitech datasets into a two-dimensional space for visualization.

From Figure 5, Figure 6 and Figure 7, it can be observed that on the MSRA25, COIL20, and Hitech datasets, the low-rank representation matrix obtained after dimensionality reduction using the ASNMF-SRP algorithm exhibits stronger discriminative power. Therefore, the ASNMF-SRP algorithm demonstrates excellent clustering performance.

4.5. Analysis of the Impact of Autoencoder-like NMF on Clustering Performance

Unlike traditional NMF, the objective function of ASNMF-SRP includes a “decoder- encoder” module. To investigate the influence of this autoencoder-like NMF architecture on clustering performance, we removed the “encoder” part in Equation (24), causing ASNMF-SRP to degenerate into ASNMF-SRP-1. The objective function of ASNMF-SRP-1 is as follows:

$$\underset{\mathit{U},\mathit{V}\ge 0}{\min }{‖\mathit{X}-\mathit{U}\mathit{V}‖}_F^2+\alpha \mathrm{Tr}\left({\mathit{V}\mathit{L}\mathit{V}}^T\right)+{‖{\mathit{X}\mathit{X}}^T-\beta {\mathit{U}\mathit{U}}^T‖}_{2,1}+\lambda \left(\mathrm{Tr}\left({\mathit{V}}^T\mathit{V}{\mathbf{1}}_n\right)-\mathrm{Tr}\left({\mathit{V}}^T\mathit{V}\right)\right)$$

(54)

From

Table 8. Clustering results of ASNMF-SRP and ASNMF-SRP-1 (mean ± standard deviation).

Dataset	ACC		ARI		NMI		PUR
	ASNMF-SRP-1	ASNMF-SRP	ASNMF-SRP-1	ASNMF-SRP	ASNMF-SRP-1	ASNMF-SRP	ASNMF-SRP-1	ASNMF-SRP
MSRA25	0.49680	0.57904	0.33356	0.44662	0.55612	0.71512	0.51656	0.61479
	±0.021	±0.045	±0.016	±0.052	±0.019	±0.026	±0.019	±0.035
Semeion	0.64724	0.68063	0.48198	0.48809	0.64292	0.63282	0.68908	0.69739
	±0.010	±0.050	±0.007	±0.034	±0.010	±0.021	±0.011	±0.031
COIL20	0.80552	0.84174	0.76313	0.80244	0.88651	0.91529	0.83587	0.86455
	±0.015	±0.012	±0.017	±0.007	±0.008	±0.006	±0.013	±0.011
COIL100	0.48007	0.64173	0.40311	0.53573	0.73255	0.83835	0.52556	0.69599
	±0.009	±0.010	±0.011	±0.016	±0.005	±0.003	±0.007	±0.006
Krvs	0.52839	0.56813	0.00393	0.01814	0.00396	0.01250	0.53339	0.56813
	±0.017	±0.003	±0.004	±0.002	±0.004	±0.001	±0.012	±0.003
Hitech	0.23301	0.25367	0.00014	0.00689	0.00893	0.01935	0.26847	0.28525
	±0.005	±0.002	±0.001	±0.000	±0.001	±0.000	±0.003	±0.001
PenDigits	0.66533	0.80442	0.53130	0.68487	0.68684	0.80120	0.69495	0.81095
	±0.040	±0.032	±0.030	±0.035	±0.020	±0.021	±0.026	±0.023
Vehicle	0.44746	0.45236	0.13445	0.12014	0.20032	0.18544	0.44888	0.45236
	±0.004	±0.002	±0.003	±0.001	±0.008	±0.000	±0.006	±0.002

, it can be seen that the clustering performance of ASNMF-SRP is significantly better than that of ASNMF-SRP-1. On the clustering evaluation metrics ACC, ARI, NMI, and PUR, it shows average improvements of 12.03%, 17.02%, 10.81%, and 10.56%, respectively. This indicates that introducing an NMF component with a structure similar to an autoencoder has a positive effect on ASNMF-SRP, effectively enhancing its clustering performance.

4.6. Analysis of the Impact of Higher-Order Graph Regularization on Clustering Performance

To further examine the influence of higher-order neighborhood relationships among samples on clustering performance, we set $w_2=0$ in Equation (17), thereby reducing ASNMF-SRP to ASNMF-SRP-2. At this point, the objective function of ASNMF-SRP-2 is as follows:

$$\begin{array}{l}\underset{\mathit{U},\mathit{V}\ge 0}{\min }{‖\mathit{X}-\mathit{U}\mathit{V}‖}_F^2+{‖\mathit{V}-{\mathit{U}}^T\mathit{X}‖}_F^2+\alpha \mathrm{Tr}\left(\mathit{V}{\mathit{L}}_1{\mathit{V}}^T\right)+{‖\mathit{X}{\mathit{X}}^T-\beta \mathit{U}{\mathit{U}}^T‖}_{2,1}\\ \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}+\lambda \left(\mathrm{Tr}\left({\mathit{V}}^T\mathit{V}{\mathbf{1}}_n\right)-\mathrm{Tr}\left({\mathit{V}}^T\mathit{V}\right)\right)\end{array}$$

(55)

From

Table 9. Clustering results of ASNMF-SRP and ASNMF-SRP-2 (mean ± standard deviation).

Dataset	ACC		ARI		NMI		PUR
	ASNMF-SRP-2	ASNMF-SRP	ASNMF-SRP-2	ASNMF-SRP	ASNMF-SRP-2	ASNMF-SRP	ASNMF-SRP-2	ASNMF-SRP
MSRA25	0.57518	0.57904	0.44437	0.44662	0.72012	0.71512	0.61354	0.61479
	±0.032	±0.045	±0.038	±0.052	±0.019	±0.026	±0.025	±0.035
Semeion	0.67803	0.68063	0.48380	0.48809	0.62883	0.63282	0.69567	0.69739
	±0.044	±0.050	±0.029	±0.034	±0.018	±0.021	±0.025	±0.031
COIL20	0.83865	0.84174	0.80155	0.80244	0.91285	0.91529	0.86194	0.86455
	±0.013	±0.012	±0.008	±0.007	±0.004	±0.006	±0.013	±0.011
COIL100	0.63818	0.64173	0.54346	0.53573	0.83103	0.83835	0.69085	0.69599
	±0.007	±0.010	±0.014	±0.016	±0.003	±0.003	±0.004	±0.006
Krvs	0.56884	0.56813	0.01850	0.01814	0.01274	0.01250	0.56884	0.56813
	±0.000	±0.003	±0.000	±0.002	±0.000	±0.001	±0.000	±0.003
Hitech	0.24744	0.25367	0.00424	0.00689	0.01893	0.01935	0.27912	0.28525
	±0.001	±0.002	±0.001	±0.000	±0.000	±0.000	±0.001	±0.001
PenDigits	0.79936	0.80442	0.67517	0.68487	0.79476	0.80120	0.80499	0.81095
	±0.029	±0.032	±0.028	±0.035	±0.017	±0.021	±0.022	±0.023
Vehicle	0.45219	0.45236	0.12018	0.12014	0.18540	0.18544	0.45219	0.45236
	±0.001	±0.002	±0.001	±0.001	±0.001	±0.000	±0.001	±0.002

, it can be observed that ASNMF-SRP exhibits superior clustering performance over ASNMF-SRP-2 on most datasets, indicating its stronger generalization ability. This also suggests that higher-order neighborhood relationships among samples can influence the clustering performance of the algorithm. However, on a few datasets (e.g., Krvs), ASNMF-SRP does not perform well. Therefore, future work should further explore how to design an optimal Laplacian matrix to effectively encode multi-order topological structure information, thereby improving the algorithm’s universality.

4.7. Robustness Analysis of ASNMF-SRP

To evaluate the robustness of ASNMF-SRP, we added salt-and-pepper noise with intensities of 10%, 20%, 30%, and 40% to the MSRA25, COIL20, and COIL100 datasets. To further compare the contributions of Equations (19) and (20) to the robustness of ASNMF-SRP, we conducted comparative experiments between ASNMF-SRP and ASNMF-SRP-3. The objective function of ASNMF-SRP-3 is as follows:

$$\begin{array}{l}\underset{\mathit{U},\mathit{V}\ge 0}{\min }{‖\mathit{X}-\mathit{U}\mathit{V}‖}_F^2+{‖\mathit{V}-{\mathit{U}}^T\mathit{X}‖}_F^2+\alpha \mathrm{Tr}\left(\mathit{V}\mathit{L}{\mathit{V}}^T\right)+{‖\mathit{X}{\mathit{X}}^T-\beta \mathit{U}{\mathit{U}}^T‖}_F^2\\ \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}+\lambda \left(\mathrm{Tr}\left({\mathit{V}}^T\mathit{V}{\mathbf{1}}_n\right)-\mathrm{Tr}\left({\mathit{V}}^T\mathit{V}\right)\right)\end{array}$$

(56)

From Figure 8, it can be seen that as the noise intensity increases, the clustering evaluation metric values of both ASNMF-SRP and ASNMF-SRP-3 show a declining trend. However, the clustering evaluation metric values of ASNMF-SRP are consistently higher than those of ASNMF-SRP-3. Therefore, adopting the $l_{2,1}$-norm to measure reconstruction errors in the “feature relationship preservation” component can enhance the robustness of ASNMF-SRP.

4.8. Parameter Sensitivity Analysis

ASNMF-SRP has three hyperparameters: the higher-order graph regularization coefficient $\alpha$, the balance parameter for feature relationship preservation $\beta$, and the regularization parameter for sparse constraint $\lambda$. In the parameter sensitivity analysis experiment, the value ranges for $\alpha$ and $\beta$ were set to $\{{10}^0,{10}^1,{10}^2,{10}^3\}$, and the range for $\lambda$ was $\{{10}^{-4},{10}^{-3}\}$. The parameter sensitivity concerning $\alpha$, $\beta$, and $\lambda$ is shown in Figure 9.

From Figure 9, we can obtain the following:

(1) On the MSRA25, COIL20, COIL100, and Hitech datasets, ASNMF-SRP demonstrates favorable clustering performance when the higher-order regularization parameter $\alpha =1000$; on the Semeion and Vehicle datasets, ASNMF-SRP exhibits good clustering performance when parameter $\alpha$ takes smaller values.

(2) With parameters $\alpha$ and $\beta$ determined, different values of the sparse constraint regularization parameter $\lambda$ significantly affect the clustering performance of the ASNMF-SRP algorithm. For example, on the COIL100 dataset, when $\lambda =0.0001$, the clustering results of ASNMF-SRP overall outperform those when $\lambda =0.001$.

(3) $\alpha$ and $\lambda$ jointly constrain the coefficient matrix $\mathit{V}$, where $\alpha$ preserves the manifold structure of samples while $\lambda$ balances the strength of sparsity, both optimizing the coefficient matrix through synergistic effects. Simultaneously, $\alpha$ and $\beta$ collaboratively optimize the local geometric structure of data in both sample space and feature space. In parameter settings, the values of $\alpha$ and $\beta$ are generally greater than that of $\lambda$ to enhance manifold structure preservation capability, while the value of $\lambda$ is kept relatively small to ensure appropriate sparse constraint intensity.

4.9. Empirical Convergence

This section will experimentally analyze the convergence of the ASNMF-SRP algorithm on 8 datasets to further verify the accuracy of the theoretical analysis in Section 3.7. In the empirical convergence experiment, the values of hyperparameters $\alpha$, $\beta$, and $\lambda$ in ASNMF-SRP remain consistent with those in Table 3.

From Figure 10, it can be observed that the ASNMF-SRP algorithm achieves convergence on all eight datasets. Specifically, on the MSRA25, Semeion, COIL20, COIL100, Krvs, Hitech, and PenDigits datasets, the ASNMF-SRP algorithm exhibits a relatively fast convergence rate during the initial iterations, with its corresponding objective function value rapidly decreasing, ultimately achieving convergence within 20 iterations. On the Vehicle dataset, the ASNMF-SRP algorithm shows a rapid decline in the objective function value during the initial iterations, but subsequently, the rate of decrease slows down, ultimately achieving convergence around 40 iterations.

5. Conclusions

This paper proposes an autoencoder-like sparse non-negative matrix factorization with structure relationship preservation (ASNMF-SRP). By integrating the principle of autoencoders, the algorithm enhances the optimization of the coefficient matrix through a linear “decoder-encoder” approach, which not only improves the factorization stability of the coefficient matrix but also further uncovers the latent representations of the original data. In the structural learning of the sample space, ASNMF-SRP effectively captures higher-order topological information between samples by constructing the optimal Laplacian matrix. In feature relationship learning, ASNMF-SRP employs the $l_{2,1}$-norm to define the reconstruction error of feature correlations between the basis matrix and the original data matrix, ensuring consistency in feature relationships between the low-dimensional space and the original high-dimensional space. Furthermore, a sparse constraint based on inner product representation is imposed on the coefficient matrix, further enhancing the clustering performance of ASNMF-SRP. Finally, comparative experiments between ASNMF-SRP and nine other advanced clustering algorithms demonstrate, as evidenced by the data in Table 4, Table 5, Table 6 and Table 7, that ASNMF-SRP achieves favorable clustering performance. In future research work, we will conduct in-depth exploration from the following three aspects: (1) To further optimize the construction process of high-order graph regularization, enhancing the model’s capability to represent complex structures; (2) to extend ASNMF-SRP into a deep NMF framework, enabling better exploration of latent features in data; and (3) to design a multi-view extension model for ASNMF-SRP, thereby broadening the applicability of the algorithm.