Hypergraph-Clustering Method Based on an Improved Apriori Algorithm

Abstract

With the complexity and variability of data structures and dimensions, traditional clustering algorithms face various challenges. The integration of network science and clustering has become a popular field of exploration. One of the main challenges is how to handle large-scale and complex high-dimensional data effectively. Hypergraphs can accurately represent multidimensional heterogeneous data, making them important for improving clustering performance. In this paper, we propose a hypergraph-clustering method dubbed the “high-dimensional data clustering method” based on hypergraph partitioning using an improved Apriori algorithm (HDHPA). First, the method constructs a hypergraph based on the improved Apriori association rule algorithm, where frequent itemsets existing in high-dimensional data are treated as hyperedges. Then, different frequent itemsets are mined in parallel to obtain hyperedges with corresponding ranks, avoiding the generation of redundant rules and improving mining efficiency. Next, we use the dense subgraph partition (DSP) algorithm to divide the hypergraph into multiple subclusters. Finally, we merge the subclusters through dense sub-hypergraphs to obtain the clustering results. The advantage of this method lies in its use of the hypergraph model to discretize the association between data in space, which further enhances the effectiveness and accuracy of clustering. We comprehensively compare the proposed HDHPA method with several advanced hypergraph-clustering methods using seven different types of high-dimensional datasets and then compare their running times. The results show that the clustering evaluation index values of the HDHPA method are generally superior to all other methods. The maximum ARI value can reach 0.834, an increase of 42%, and the average running time is lower than other methods. All in all, HDHPA exhibits an excellent comparable performance on multiple real networks. The research results of this paper provide an effective solution for processing and analyzing large-scale network datasets and are also conducive to broadening the application range of clustering techniques.

1. Introduction

With the explosive growth of information, the huge amount of data generated [1] has permeated every industry and field. Moreover, most of these data presented high dimensions and complex structures. How to handle these massive and complex data has become an urgent challenge in the field of machine learning at present. Clustering is one of the most important methods in machine learning, which reasonably assigns data into a cluster by selecting a measurement method [2]. Traditional clustering methods are mainly using classical machine learning theoretical knowledge to conduct research, such as partition-based clustering methods [3] and hierarchy-based clustering methods [4], etc. With the progression of time, although many scholars improve the traditional clustering algorithm on this basis, due to the special characteristics of complex data, they often cannot achieve ideal clustering results, which is mainly reflected in poor clustering division results [5]. Due to the fact that the graph-based data analysis method can identify well-connected vertices clusters [6], this makes up for the shortcomings of ordinary high-dimensional data clustering. Therefore, clustering analysis through graphs has become a more common processing method. However, ordinary graphs cannot comprehensively describe various complex relationships in data, such as the World Wide Web, airline networks, etc. These networks often have intricate internal structures [7,8,9,10]. Inspired by this, hypergraphs have emerged. Hypergraphs [11] are mainly used to study the generalization of multiple relationships in graph topology structures and are widely used in many fields of information technology. One popular direction is to use hypergraphs for clustering analysis [12] and to effectively represent high-order correlations between data in a visualized form using hypergraphs [13].

In the contemporary period, many new clustering algorithms have been proposed to handle complex data by constructing hypergraphs. Strehl et al. [14] transformed the clustering problem into a hypergraph-partitioning combinatorial optimization problem, and proposed three consensus functions for better clustering. Yang et al. [15] systematically discussed cluster ensembles, compared the advantages and disadvantages of traditional clustering algorithms, and extended their analysis to hypergraph partitioning. Based on the evolutionary model of the hypernetwork dynamics process, Suo et al. [16] provided a comprehensive overview of the future development directions in the field of hypergraphs. Tian et al. [17] introduced knowledge hypergraphs using heterogeneous hypergraphs, constructed a three-layer architecture to better represent and extract hyper-featured relationships, and modeled hyper-relation data. Liu et al. [18] constructed a hypergraph to embed sample representation and proposed an optimized constrained clustering algorithm with upper and lower limits on the number of clusters. All these studies have demonstrated the superiority and universality of hypergraph-clustering methods.

Due to the complexity and variability of today’s data, it is still necessary to adopt more scientific and objective methods in constructing hypergraphs. Association rules are commonly used, as they can acquire frequent itemsets and association rules to clarify the relationships between data. Wei et al. [19] proposed the HOT algorithm combining outlier detection based on the hypergraph model and association rules to calculate the support degree, membership degree, and other information for each point. However, it may suffer from local optima issues. Cui et al. [20] comprehensively discussed several applications of hypergraphs in data mining, including association rules, clustering, and spatial data mining. Kadir et al. [21] constructed a hypergraph model and investigated the association relationships in hypergraphs based on the hypergraph-partitioning algorithm hMETIS and the association rule algorithm Apriori. However, the efficiency of the Apriori algorithm [22] is relatively low due to its iterative method of searching frequent itemsets in a bottom-up manner, making it unable to analyze rare information. Lately, many fruits related to intelligent clustering technology have emerged. Althuwaynee et al. [23] combined machine learning with geospatial technology by studying landslides using nonlinear clustering t-SNE and Apriori association rule mining. Esmaeili et al. [24] proposed a combination method based on the fuzzy firefly algorithm and random forest and applied it to clustering in wireless sensor networks (WSNs). Zhao et al. [25] proposed several higher-order centralities in high-order complex networks to reflect the impact of ranking on clustering and propagation. Chen et al. [26] proposed a DPC algorithm based on Chebyshev inequality and differential privacy and discussed the data privacy protection function. These studies have expanded our research ideas on clustering.

Based on these studies, we propose a new clustering method based on improved Apriori and hypergraph partitioning. This method first uses t-SNE to reduce the dimensionality of the dataset and combines the improved Apriori algorithm with the hypergraph construction method to construct a hypergraph for model clustering. Then, it employs the dense subgraph partition algorithm based on multi-level hypergraph partitioning to divide the hypergraph into numerous sub-clusters. After merging the sub-clusters, the final clustering results are obtained. Finally, various methods are compared on multiple datasets through clustering evaluation indicators, and the operating efficiency of all methods is analyzed, which demonstrates the effectiveness of the proposed method from many aspects.

The specific contributions of this paper are as follows:

(1)

This paper extends association rules to hypergraphs, then proposes a hypergraph-partitioning high-dimensional data-clustering method based on the improved Apriori algorithm (HDHPA), providing a solution for identifying clustering results of high-dimensional data.

(2)

In order to avoid the generation of redundant rules and improve the mining efficiency, the parallel mining of different frequent itemsets is used to obtain hyperedges with corresponding ranks.

(3)

The HDHPA method improves clustering accuracy by constructing a hypergraph structure of high-dimensional data. In order to demonstrate the superiority of the method, this paper conducted numerous experiments on multiple datasets.

The remaining sections of this paper are organized as follows. Section 2 affords some theoretical foundations correlated to the proposed methods. Section 3 conducts the HDHPA method and its three stages. Section 4 analyzes the experiments from different perspectives to validate the feasibility and superiority of the method. Section 5 summarizes the findings and prospects for future research directions.

2. Related Theoretical Foundations

2.1. The Concept of Hypergraph

The hypergraph H can be defined as a binary group $H=\left(V,E\right)$, where V is a set of vertices $V=\left\{v_1,v_2,\dots ,v_n\right\}$ and E is a set of hyperedges $E=\left\{e_1,e_2,\dots ,e_m\right\}$ are finite sets, and $e_i\ne \varphi \left(i=1,2,\dots ,m\right)$, ${\cup }_{i=1}^m{e}_i=V$. The number of vertices contained in the hyperedge is called the rank. For $r\left(H\right)=max\left|e_i\right|$, it is called the upper rank of the hypergraph, and for $s\left(H\right)=min\left|e_i\right|$, it is called the lower rank of the hypergraph. If $r\left(H\right)=s\left(H\right)$, then H is a uniform hypergraph. In hypergraph H, the number of hyperedges containing node $v_i$ is called the hyperdegree, denoted as $d_H\left(v_i\right)$.

The incidence matrix $I\left(H\right)$ of hypergraph H is a matrix where the rows correspond to the vertices of H and the columns correspond to the hyperedges. The adjacency matrix $A\left(H\right)$ of hypergraph H is a square symmetric matrix, where the element $a_{ij}$ represents the number of hyperedges containing nodes $v_i$ and $v_j$; the diagonal entries of $A\left(H\right)$ are zero. For a set $A\subseteq V$, $H_A=(A,\{e_i\cap A|e_i\in E\wedge e_i\cap A\ne \varnothing \})$ is called the sub-hypergraph of the hypergraph $H=\left(V,E\right)$, or rather, the hypergraph on the vertices subset of H is called the sub-hypergraph of H, such that for each hyperedge in the sub-hypergraph, there exists only one hyperedge in H that contains it.

Figure 1 shows an example of a hypergraph. In this hypergraph, vertices with a degree of 1 are $v_1,v_3,v_4,v_7\mathrm{and}{v}_8$, vertices with a degree of 2 are $v_2\mathrm{and}{v}_5$, the vertex with a degree of 3 is $v_6$, and the four hyperedges in Figure 1 do not all contain the same number of vertices, making it a non-uniform hypergraph.

2.2. Apriori of Association Rules

Association rules [27] are mainly used to find rules of the form $A⟹B$ from a given dataset. This process can be decomposed into two sub-processes: determining frequent itemsets and establishing association rules. Apriori, as a classic association rule algorithm, generates frequent itemsets based on the prior principle and extracts high-confidence rules with only one item in the consequent based on confidence pruning. (In this paper, according to the dataset characteristics, we used the trial-and-error sampling method to determine the size of the support threshold). The concrete algorithm steps are as follows:

• Scan the dataset L, calculate the support of each item, discard items that do not meet the support threshold, and generate one-dimensional frequent itemsets.
• Iteration: generate two itemsets through the one-dimensional itermsets, discard items that do not meet the support threshold, and generate two-dimensional frequent itemsets.
• Repeat, establish k-dimensional candidate itemsets, discard items that do not meet the support threshold, and generate k-dimensional frequent itemsets.
• Prune each k-itemset; count and retain the association rule itemset that satisfies the set confidence threshold.

Let I be a set of N different items, denoted as $I=\left\{i_1,i_2,\dots ,i_N\right\},$ referred to as itemset. The length of an itemset is the number of items it contains. A k-itemset represents an itemset with a length of k, also known as dimension. $T=\left\{t_1,t_2,\dots ,t_n\right\},$ is a transaction set, where each transaction $t_i$ is a set of items, $t_i\subseteq I$. An association rule has the following form: $A⟹B$, where $A,B\subset I$ and $A\cap B=\varnothing$. For a given itemset $X\subset I$, the number of transactions containing X is called the support count of itemset X, denoted as $\sigma \left(X\right)$, which is defined as $\sigma \left(X\right)=\left|\left\{t_i\left|X\subseteq t_i, t_i\in T\right.\right\}\right|$. Therefore, the support of the rule $A⟹B$ is defined as $support\left(A\Rightarrow B\right)=\sigma \left(A\cup B\right)/n$. The confidence is defined as $confidence\left(A\Rightarrow B\right)=\sigma \left(A\cup B\right)/\sigma \left(A\right)$; the higher the value of confidence, the greater the credibility of B inferring the appearance of A. In order to avoid ignoring the support of the itemsets in the rule posterior, the definition of lifting degree is introduced as

$$lift(A\Rightarrow B)=\frac{confidence(A\Rightarrow B)}{support(B)}$$

(1)

$lift\left(A\Rightarrow B\right)$ reflects the degree of influence of the occurrence of A on the occurrence of B in the hyperedge $e_i$, and is a correlation metric. If the lift degree l > 1, A and B are positively correlated; if l = 1, A and B are independent of each other and there is no correlation; if l < 1, A and B are negatively correlated.

2.3. Hypergraph Model Definition with Association Rules

Combining the descriptions in Section 2.1 and Section 2.2, we can correspond the hypergraph $H=\left(V,E\right)$with association rules. For the hypergraph $H=\left(V,E\right)$, each vertex $v_i$ corresponds to an item in the dataset, and the hyperedge $e_i\left(i=1,2,\dots ,m\right)$ is a transaction that contains an itemset, the vertex set denoted as $V(e_i)$ (also called V), and the hyperedge set denoted as $E\left(e_i\right)$. Therefore, the support of the itemset X is

$$support(X)=\sigma (X)/n$$

(2)

where the support count of itemset X ($X\subseteq E)$ is $\sigma \left(X\right)=\left|\left\{\left.t_i\right|X\subseteq t_i,t_i\in T\right\}\right|$.

Let the association rule r discovered in itemset X is $A\Rightarrow B$, where $A\subseteq X, B\subseteq X$, then the confidence of this association rule is found by

$$confidence(r)=\sigma (A\cup B)/\sigma (A)$$

(3)

If the itemset X satisfies the minimum support threshold, the itemset X is denoted as a frequent itemset and a hyperedge $e_i$ is constructed by a frequent itemset, so the weight of hyperedge $e_i$ is defined as follows:

$$weight(ei)={\displaystyle \sum _{r}confidence(r)/R}$$

(4)

where R is the number of association rules discovered in itemset X.

Figure 2 shows an elemental example of constructing a hypergraph based on association rules, where the dataset is represented by binary 0/1, and the minimum support is set to 60%.

2.4. DSP Algorithm in Hypergraph Partitioning

In this section, we introduce a hypergraph-partitioning algorithm based on dense subgraph partition, abbreviated as DSP [28]; this algorithm can spontaneously and effectively partition hypergraphs into dense subgraphs, called DPs. Based on the idea of divide-and-conquer, DSP is suitable for parallel processing and can be applied to large-scale hypergraphs efficiently. Therefore, we used it to obtain enough sub-clusters.

A permutation P of a set of elements represents a specific order in which the elements are arranged according to a certain rule, and its sub-permutation R refers to a continuous sequence of P. For a hypergraph $H=\left(V,E\right)$, the density $\rho \left(e_i\right)=weight\left(e_i\right)/\left|V\left(e_i\right)\right|$, which is the ratio of the weight of the hyperedge $e_i$ to the number of vertices included. Similarly, the density of hypergraphs can be obtained. The sub-hypergraph with the maximum density in hypergraph H is called the densest sub-hypergraph. There is only one densest sub-hypergraph with the largest number of vertices, which is called the kernel sub-hypergraph. When conditions are set and multiple sub-hypergraphs that meet the conditions have the highest density, the sub-hypergraph with the highest number of vertices is called the conditional kernel sub-hypergraph. If $H_u$ is the conditional kernel sub-hypergraph of $H_s$, then $H_u$ is partitioned into the maximum number t of non-connected sub-hypergraphs, represented as $DP\left(H_U|H_S\right)=\left\{H_{U_1},\dots ,H_{U_t}\right\}$. For the dense sub-hypergraph partition of hypergraph H, it is defined as follows:

$$DSP(H)=〈DP(H_{v_1}|\varnothing ),\dots ,DP(H_{v_i}|H_{{\cup }_{j=1}^{i-1}{v}_j}),\dots ,DP(H_{v_{\mathrm{n}}}|H_{{\cup }_{j=1}^{\mathrm{n}-1}{v}_j})〉$$

(5)

where ${\cup }_{i=1}^{n-1}{v}_i=v$ and $H_{v_1}$ is the kernel sub-hypergraph of H. $H_{v_i}(i>1)$ are conditional kernel sub-hypergraphs conditioned on the sub-hypergraphs $H_{{\cup }_{j=1}^{j-1}{v}_j}$.

Based on the above, the DSP partition algorithm includes two layers. First, the hypergraph H is continuously partitioned into a series of conditional kernel sub-hypergraphs, $\langle H_{V_1},\dots ,H_{V_n}\rangle$. The partition of vertices set V is represented as $\mathsf{\Psi }\left(V\right)=\langle v_1,\dots ,v_n\rangle$. Then, each sub-hypergraph $H_{v_i}$ is partitioned into dense sub-hypergraphs. $\mathsf{\Psi }\left(V\right)$ defines the order of partitioning the vertices set V. If there are |V| permutations of V, there must be one permutation that satisfies the order of vertices in $\mathsf{\Psi }\left(V\right)$, denoted as $\mathsf{\Theta }\left(H\right)$. Based on $\mathsf{\Theta }\left(H\right)$, it is easy to find the dense sub-hypergraphs of the partition. Therefore, the algorithm for partitioning DSP can be described as follows:

The time complexity of Algorithm 1 Is $\mathrm{O}\left(\tau d{n}_e{\left(n_v+n_e\right)}^2\right)$, where τ is the number of iterations in the algorithm and $n_v$ and $n_e$ are the number of vertices and hyperedges of the dense sub-hypergraph with maximum size, respectively.

Algorithm 1: Partitioning DSP algorithm

Input: Hypergraph H and an initial permutation P Output: DSP(H) 1:  Apply min-partition () on P to get $MP\left(P\right)$ 2:  Set $\mathsf{\Omega }=MP\left(P\right),\tilde{\mathsf{\Omega }}=\varnothing$ and $\tilde{P}=P$ 3:  Repeat 4:   for each $R\in MP\left(P\right)$ do 5:    if $R\notin \tilde{\mathsf{\Omega }}$ then 6:     Apply permutation-reorder () on R to get $\tilde{R}$ 7:     if $\tilde{R}\ne R$ then 8:      Replace R by $\tilde{R}$ in $\tilde{P}$ 9:      Apply min-partition () on $\tilde{R}$ to get $MP\left(\tilde{R}\right)$ 10:       Replace R by $MP\left(\tilde{R}\right)$ in $\mathsf{\Omega }$ 11:    end if 12:   end if 13:  end for 14:  Apply min-merge () on $\mathsf{\Omega }$to get $R\in MP\left(\tilde{P}\right)$ 15:  Set $\tilde{\mathsf{\Omega }}=MP\left(P\right)$, $P=\widehat{P}$ and $MP\left(P\right)=MP\left(\widehat{P}\right)$ 16:  until P does not change 17:  For each $R\in MP\left(P\right)$, apply disjoint-partition () on $H_R$ 18:  return DSP(H) by Equation (5)

3. Method Design of HDHPA

3.1. Basic Ideology of HDHPA

This paper proposes a new complex data clustering method called the high-dimensional data clustering method based on hypergraph partitioning using an improved Apriori algorithm (HDHPA) based on an improved Apriori algorithm and hypergraph-partitioning model. First, the method constructs a hypergraph based on the improved Apriori algorithm for association rule mining and then uses the DSP algorithm for multi-level hypergraph partitioning to partition the hypergraph into many subclusters. Finally, the subclusters are merged to obtain the final clustering results. Although previous studies, such as those in [17,18], have utilized hypergraphs to extract data information, there are limitations to high computational complexity. For example, the authors of [23] used the Apriori algorithm to determine the relationship between factors, but they did not improve the algorithm, which has problems with multiple iterations.

We use t-distributed Stochastic Neighbor Embedding (t-SNE) [29] for the preprocessing and feature extraction of the original high-dimensional data. After obtaining low-dimensional data, HDHPA discovers clusters in the dataset through three stages: in the first stage, the improved Apriori algorithm is parallelized to speed up the mining of association rules and construction of the hypergraph model; in the second stage, the dense sub-hypergraph partitioning algorithm is applied to partition the hypergraph into many relatively small dense sub-hypergraphs (or subclusters); in the third stage, two evaluation functions (Conformance function C and Aggregation function A) are used to merge dense sub-hypergraphs and obtain clustering results.

3.2. The Stage of Constructing a Hypergraph

During the construction of the hypergraph stage, the goal is to find sub-hypergraphs that meet clustering conditions and are mutually connected. Based on the generated frequent sub-hypergraphs, the relationships (or hyperedges) between the entities (or vertices) of the encapsulated objects are established. To ensure the objectivity of selecting frequent sub-hypergraphs, we decided to follow the structure of the Apriori algorithm to find frequent itemsets. To avoid the additional computational burden, we improved it based on discovering frequent itemsets with the Apriori algorithm, which can maximize the avoidance of excessive iterations and achieve effective pruning with minimal time complexity, compared with other improved association rule algorithms (such as FP-stream [30] and EDFS [31]).

Based on this idea, in the hypergraph construction stage of the HDHPA method, the vertices correspond to the itemset containing transactions; a set of hyperedges containing multiple hyperedges in a node is called an itemset. At first, this stage of the method enumerates all frequent single-hyperedge and double-hyperedge sub-hypergraphs based on Equation (2). Then, parallel retrieval is performed based on these two sets. In each iteration round, candidate sub-hypergraphs are first generated, with a size one edge larger than the previous frequent sub-hypergraph (line 7 of the algorithm). Next, the support count of each candidate item is calculated, measuring the correlation between hyperedges by Equation (3), and hyperedges that meet the threshold are merged into sub-hypergraphs. Finally, sub-hypergraphs that do not satisfy the prior principle are pruned to generate hypergraphs by Equation (1) (lines 13–16). The discovered frequent sub-hypergraphs satisfy the downward closure property of the support condition, making the constructed hypergraph robust to noisy data.

The method for constructing the hypergraph stage is shown in Algorithm 2 below. Given a minimum support threshold count of $\sigma$ to construct the hypergraph, $C^k$ is a candidate set with k hyperedges, then $H_C{}^k=\left(L,C^k\right)$ is a sub-hypergraph with k candidate sets. A hyperedge $e_i$ that meets the minimum support is called a frequent itemset $F\left(e_i\right)$, then $H_F{}^k=\left(L^{\prime },F\right)$ is a sub-hypergraph with k frequent sets, where $L^{\prime }$ is a set of frequent transactions and F is a set of $F\left(e_i\right)$. Sparse graphs are used to store input transactions, intermediate candidates, and frequent sub-hypergraphs. This representation saves memory and speeds up computation. The topology, hyperedges, and vertex labels of each frequent sub-hypergraph are randomly selected, resulting in a complete set of valid candidate items with no duplication. The weights adhere to a unit-mean exponential distribution, and the total weight of all frequent sub-hypergraphs is normalized to 1. Based on the previous analysis, the time complexity of Algorithm 2 is$\mathrm{O}\left(N{C}^k\log \left(k\right)\right)$.

Algorithm 2: HDHPA method for constructing the hypergraph stage

Input: A set of all items in dataset L: $I=\left\{i_1,i_2,\dots ,i_N\right\}$ Output: $H_F{}^1,H_F{}^2,\dots ,H_F{}^k$, $H_C{}^k.count$ 1: Compute $H_F{}^1\leftarrow \left\{i|i\in I\wedge \sigma \left(\left\{i\right\}\ge N\times minsup\right)\right\}$ by Equation (2) 2: Compute $H_F{}^2\leftarrow \left\{i|i\in I\wedge \sigma \left(\left\{i\right\}\ge N\times minsup\right)\right\}$ by Equation (2) 3: Set $H_F{}^1\leftarrow detect all frequent 1-subhypergraphs in V$ 4: Set $H_F{}^2\leftarrow detect all frequent 2-subhypergraphs in V$ 5: $k\leftarrow 3$ 6: while $H_F{}^{k-1}\ne \varnothing$ do 7:  $k\leftarrow k+1$ 8:  $C^k\leftarrow genCandidate\left(H_F{}^{k-1}\right)$ by Equation (3) 9:  $C^k\leftarrow pruneCandidate\left(C^k,H_F{}^{k-1}\right)$ 10:  for each candidate $H_C{}^k\in C^k$ do 11:   $H_C.count\leftarrow 0$ 12:   for each transaction $v\in V$ do 13:    $C^v=subset\left(C^k,v\right)$ by Equation (1) 14:    if candidate $H_C{}^k$ is included in transaction v then 15:    $H_C{}^k.count\leftarrow H_C{}^k.count+1$ 16:  $H_F{}^k\leftarrow \left\{H_C{}^k\in C^k\wedge \sigma \left(H_C{}^k\right)/N\ge minsup\right\}$ 17: until $H_F{}^k\ne \varnothing$ 18: return $H_F{}^1,H_F{}^2,\dots ,H_F{}^k$

3.3. Hypergraph Partitioning Stage

After constructing the hypergraph, we use the hypergraph-partitioning algorithm DSP introduced in Section 2.4 to partition the hypergraph. The DSP algorithm adopts the divide-and-conquer approach to perform parameter-free partitioning of the hypergraph from top to bottom, resulting in relatively low time and space complexity. Due to the uniqueness of conditional kernel sub-hypergraphs and their non-intersecting partitions, DSP is unique. A denser sub-hypergraph is more likely to represent a true cluster, while a less dense sub-hypergraph is usually composed of outliers. The result of the DSP algorithm is an ordered list of dense sub-hypergraphs with decreasing density, which reveals all potential clusters and outliers. Therefore, it can achieve good clustering results even if there are a lot of outliers.

The algorithm first calculates the weight of each hyperedge in the hypergraph, and then divides the hypergraph into a sequence of conditional kernel sub-hypergraphs in decreasing order, and calculates the corresponding density. Then, each conditional kernel sub-hypergraph is divided into disjoint dense sub-hypergraphs by a non-overlapping partitioning algorithm. The dense sub-hypergraphs generated using DSP usually have strong connections between vertices and can be considered as potential clusters to obtain clustering results [32].

3.4. Dense Sub-Hypergraph Merging

Once the entire hypergraph is partitioned into k parts, we use the following cluster fitness criterion to eliminate bad clusters. Let $v_i$ be the set of vertices representing connected hyperedges, and Z be the set of vertices representing the partition. The degree of conformance function C that measures the fitness of the partition Z is defined as follows:

$$Conformance(Z)=\frac{{\displaystyle {\sum }_{v_i\subseteq Z}Weight(v_i)}}{{\displaystyle {\sum }_{|v_i\cap Z|>0}Weight(v_i)}}$$

(6)

The conformance function measures the ratio of $v_i$ weights within a partition to any vertices’ hyperedge weights within that partition. A higher value indicates that vertices within the partition are more likely to be retained in the current cluster. Partitions with a degree of conformance greater than a given threshold are considered clusters with good clustering effects. We set the degree of conformance threshold to 0.1 (reasonably adjusting this threshold can effectively reduce the number of un-clustered points in the clustering results) so that this function can be easily combined with Apriori-based partitioning algorithms, such that only when the conformance of a partition is lower than a given threshold is the cluster further divided, until all such clusters are found.

After finding the clusters, each cluster needs to be checked to filter out vertices that are not highly connected to other nodes in the cluster. Based on the ratio of the hyperdegree of node $v_j$ in the partitioned vertex set Z to the other vertices in the same cluster, the aggregation function A is defined as follows:

$$Aggregation(v_j,Z)=\frac{\left|d_H(v_j)\right|}{\left|d_H(Z)\right|}$$

(7)

The higher the value of aggregation, the more connected hyperedges there are, which connect most of the vertices in the cluster. Points with aggregation values greater than a given threshold are considered to belong to the cluster; the rest of the vertices are discarded from the cluster. In this paper, we set the aggregation threshold to 0.1.

4. Experiments and Analysis

4.1. Dataset Description

The HDHPA method was validated for clustering problems using 6 different protein–protein interaction (PPI) network datasets and the MNIST dataset.

• Protein–protein interaction network: The vertices, attribute values, and edges correspond to proteins, GO (gene ontology [33]) terms, and protein–protein interactions, respectively. The GO terms mainly include molecular functions, cellular components, and biological processes. The experiments selected 6 PPI network datasets from real protein interaction network datasets [34], including FAA4 (from yeast), Natoc_0297 (from Cryptococcal Natrophomonas), 16 items (from Nattococcus), HOXA10 (from chromosome 7), NNMT (from cancer cells), and MBP (from Escherichia coli). Partial PPI networks constructed from the datasets are shown in Figure 3.
• MNIST: A classic handwritten digit recognition dataset, consisting of 60,000 grayscale images divided into 10 classes from “0” to “9”. In this study, 1000 samples were selected from the dataset for experimentation.

In Figure 3, the larger the blue vertices, the more proteins it interacts with, and conversely, the smaller the vertices, the fewer proteins it interacts with in the network [35]. Edges represent protein–protein associations, indicating that they are clear and meaningful, that is, proteins work together to share functions, but this does not necessarily mean that they physically bind to each other [36]. Colored edges represent the predicted degree of protein–protein interaction, increasing from orange-yellow-green-blue-purple (more about PPI network supplements as shown in Appendix A), indicating that the closer the edge color is to purple, the more homogeneous features it contains and the more similar biological information it contains. Unlike the MNIST, we can see from Figure 3 that the PPI network basically follows a power law distribution, with more sparse edge structures that are closer to real networks to verify the robustness of the HDHPA from multiple perspectives.

4.2. Clustering Evaluation Metrics

In order to quantitatively estimate the effectualness of clustering in a computational model, the performance was assessed using different evaluation metrics. We used the values of the Fowlkes and Mallows Index (FM-index) [37] and the Adjusted Rand Index (ARI) [38]. Before introducing these two types of evaluation metrics, let’s first introduce the relevant terms involved. Consider a dataset with n points. For binary classification, the rare class is referred to as the positive class, and the majority class is referred to as the negative class. The confusion matrix, summarizing the correct and incorrect predictions of the classification model, is shown in

Table 1. Confusion matrix for non-equally important two-class classification issues.

		Predicted Class
		+	−
Actual class	+	TP 1	FN 2
	−	FP 3	TN 4

¹ TP (True Positive): the number of correctly predicted positive sample pairs. ² FN (False Negative): the number of positive sample pairs incorrectly predicted as negative classes. ³ FP (False Positive): the number of negative sample pairs incorrectly predicted as positive classes. ⁴ TN (True Negative): the number of correctly predicted negative sample pairs.

below:

From the above, clustering usually uses indicators to evaluate the degree of consistency between clustering results and actual data. For network data, we want to assign two nodes to the same cluster if and only if they are similar. Therefore, in cluster evaluation metrics, TP represents the assignment of two similar nodes to the same cluster. TN represents the assignment of two dissimilar nodes to different clusters. FP represents the assignment of two dissimilar nodes to the same cluster. FN represents the assignment of two similar nodes to different clusters.

The equation of FM-index value [37] is

$$FM-index=\sqrt{\frac{TP}{TP+FP}\cdot \frac{TP}{TP+FN}}$$

(8)

The equation of ARI value [38] is

$$ARI=\frac{RI-E[RI]}{\max (RI)-E[RI]}$$

(9)

where $RI=\left(TP+TN\right)/C_2^{n_{samples}}$; C is the total number of possible element pairs that can be formed in the dataset. The value range of the FM-index is [0, 1]. The larger the value of the FM-index, the better the clustering effect, indicating a more accurate representation of the factual situation. The value range of ARI is [−1, 1]. Specifically, a value of 1 indicates that the clustering result matches the fact almost perfectly, a value of 0 indicates that the clustering performance is the same as that generated randomly, and a value of −1 indicates that the clustering result completely contradicts the fact.

4.3. Data Processing

To objectively and comprehensively verify the performance of the HDHPA method, all experiments in this paper were conducted on the same operating system, based on a Python 3.8.13 parallel computing environment. In the data preprocessing stage, duplicate hyperedges were removed and the original data were dimensionally reduced using t-SNE, with a maximum iteration setting of 3000. In PPI network datasets, we set the protein set as the vertex set and protein–protein interactions as hyperedges, then constructed a hypergraph based on this presumption.

Table 2. PPI network datasets were used for the experiments.

Dataset	Number of Proteins	Number of Interactions	Attribute Values
FAA4	789	636	1140
Natoc_0297	991	1031	1227
16 items	1106	2611	2115
HOXA10	2151	4367	2446
NNMT	2612	5258	2676
MBP	3121	7423	3235

summarizes some statistics about the PPI network datasets:

4.4. Parameter Settings

The purpose of this section was to ensure the robustness of experimental performance, study parameter sensitivity, and understand how changes in the support count $\sigma$ will affect the different weights between data relationships and hyperedge selection, as well as how it will affect clustering results. Specifically, the FAA4 dataset was used for the experiment, and the results are shown in Figure 4. It can be seen that when the parameter $\sigma$ is relatively small compared to k, the number of sub-hypergraphs generated by DSP partitioning the hypergraph is also small, and the dense sub-hypergraphs generated by DSP partitioning will be particularly large, so the number will also be correspondingly reduced. In addition, the number of dense sub-hypergraphs is linearly related to σ, and when σ > 5, the number of dense sub-hypergraphs generated by DSP is relatively robust. Therefore, for simplicity, the minimum σ is set to 7 in this experiment.

4.5. Experimental Comparison Methods

Since the HDHPA method combines hypergraph partitioning and clustering, we selected two classic methods for partitioning and clustering, and combined them pairwise as comparisons.

• k-medoids [39]: A classic clustering method based on representative objects that select objects located at the center of clusters as center points. It is more suitable for processing small datasets with noise and outliers.
• hMETIS [40]: A hypergraph-partitioning method based on the METIS graph-partitioning software package version 5.1.0 that can efficiently partition hypergraphs through k-way multilevel partitioning.

Five methods, including HDHPA, use seven datasets for clustering comparison: k-medoids + hMETIS (kh), k-medoids + DSP (kD), Apriori + hMETIS (Ah), Apriori + DSP (AD), and HDHPA. Since the cluster centers of the k-medoids method are randomly defined, each method was run 100 times, and the average of these 100 results was taken as the final result.

4.6. Experimental Comparison Results

4.6.1. Comparison of Experimental Results on PPI Networks

This section applied five hypergraph-clustering methods to six protein–protein interaction network datasets of varying sizes. After clustering the selected datasets, we analyze their enrichment in GO terms associated with biological processes. The results are shown in Figure 5.

In the enrichment analysis shown, the size of the circle represents the degree of significance of the enrichment and the color of the circle gradually decreases in significance from red to blue. Due to the small p-value, this paper chose the scale and color based on the $-\log \left(P\right)$ value of each term’s importance. Comparing six PPI network datasets, it can be seen from Figure 5 that they exhibit similar characteristics. GO terms that cluster more similarly to each other are also closer in space, and vertices with richer colors show changes in enrichment.

Then, we summarized the information of six PPI network datasets, seen in in

Table 3. Summary information of 6 protein–protein interaction networks.

Dataset	Avg. Clustering Coefficient	Avg. Number of Neighbors	Network Density	Network Heterogeneity	Network Centralization
FAA4	0.651	13.978	0.155	0.562	0.341
Natoc_0297	0.701	22.659	0.252	0.636	0.470
16 items	0.695	11.638	0.112	0.740	0.180
HOXA10	0.630	44.596	0.297	0.584	0.489
NNMT	0.652	27.648	0.307	0.566	0.493
MBP	0.631	23.521	0.196	0.677	0.445

. We can see that the Natoc_0297 dataset has the highest average clustering coefficient, the HOXA10 dataset has the maximum average number of neighbors, the NNMT dataset has the highest network density, the 16 items dataset has the maximum value of network heterogeneity, and the NNMT dataset has the highest network centrality. According to the relevant theoretical knowledge of PPI networks [41], the clustering results of the Natoc_0297 and HOXA10 datasets are easier to distinguish. For the FAA4, 16 items, NNMT, and MBP datasets, there may be local optima in the clustering results of the dataset, but this situation is not within the scope of consideration.

The evaluation indices FM-index and ARI values of the clustering results are summarized in

Table 4. FM-index values of clustering results using 5 methods (unit %).

Method	Dataset
	FAA4	Natoc_0297	16 Items	HOXA10	NNMT	MBP
kh	70.97	66.19	55.18	60.77	50.15	62.95
kD	77.80	74.16	50.29	51.40	46.64	61.01
Ah	44.14	47.51	24.34	38.51	19.33	49.20
AD	56.47	40.58	21.90	25.11	26.72	33.17
HDHPA	80.46	73.39	70.13	65.60	49.55	73.18

and

Table 5. ARI values of clustering results using 5 methods (unit %).

Method	Dataset
	FAA4	Natoc_0297	16 Items	HOXA10	NNMT	MBP
kh	77.62	74.50	34.98	56.97	46.95	67.06
kD	82.83	78.15	47.60	70.47	49.92	75.40
Ah	34.34	33.56	13.47	18.52	14.23	29.60
AD	58.67	50.91	38.12	46.33	58.95	48.71
HDHPA	81.13	80.60	49.29	67.81	57.56	79.88

.

As shown in Table 4 and Table 5, For the FAA4 dataset, the FM-index value of the HDHPA method is higher than other methods, up to 0.805. The ARI value of the kD method is higher than other methods and is very close to the ARI value of the HDHPA method, far exceeding other methods. In the Natoc_0297 dataset, the FM-index value of the kD method is higher than other methods. The FM index value of the HDHPA method is very close to it, with values of 0.741 and 0.734, respectively, and the ARI value of the HDHPA method is higher than other methods. For the 16-item dataset, the FM-index value and the ARI value of the HDHPA method are the largest. In the HOXA10 dataset, the FM-index value of the HDHPA method far exceeds that of other methods, but the ARI value of the kD method is greater than that of the other methods. In the NNMT dataset, the FM-index value of the kh method is the largest. The HDHPA method values of the FM-index are very close to it, 0.502 and 0.496, respectively. The ARI values of the AD and the HDHPA are very close to eachother, 0.590 and 0.576, respectively. For the MBP dataset, the FM-index and ARI values of the HDHPA are higher than those of other methods, 0.732 and 0.799, respectively.

The experimental results show that no method performs significantly better than others. In most cases, our proposed HDHPA method combined with clustering produces the best results. The clustering results of the other four comparison methods are similar for the 16-item dataset, indicating that the precision gain is relatively limited in datasets with high clustering coefficients. The kD method produces better clustering results for the HOXA10 and Natoc_0297 datasets than the Ah and AD methods, indicating that when there is more noise and isolated points, the efficiency of the k-medoids method is higher than that of the Apriori. In most cases, the results of the Ah method are more sensitive to data types and less robust than the other four clustering methods.

4.6.2. Comparison of Experimental Results on MNIST

To verify the universality and accuracy of the HDHPA method from multiple perspectives, this section selects the MNIST dataset and compares it with the same method combination as the benchmark. The FM-index and ARI values of the clustering results are summarized in

Table 6. The clustering evaluation index of each method for MNIST dataset (unit %).

Method	MNIST
	FM-Index	ARI
kh	45.96	43.06
kD	46.52	48.40
Ah	63.97	57.24
AD	39.51	34.22
HDHPA	70.60	83.37

.

As shown in Table 6, on the MNIST dataset, the FM-index and ARI values of the HDHPA method are much higher than those of other methods, demonstrating superior performance. In addition, compared to 6 PPI network datasets, the HDHPA method increased clustering by up to 0.834. The Ah method increased in both FM-index and ARI values, with an increase of at least 14.77% and 22.9%, respectively. The result indicates that methods based on support for finding frequent itemsets are more suitable for clustering text bytes in practice, possibly because the Apriori algorithm reduces the weight of noisy data when dealing with the same hyperedge with different weights. Based on the experimental results in the first two sections, we can find that the method proposed in this paper has more universality and effectiveness. Furthermore, we compared it with the latest research, such as using the same evaluation indicators as [26], and the evaluation indicator values obtained from the two experiments were very close, reflecting the effectiveness of the proposed method in this paper.

4.6.3. Running Time Comparison

In this section, we compare the computational efficiency of hypergraph-clustering methods by comparing the running time complexity of all experiments. All experiments were conducted on the same server with the same configuration: Windows 10 Professional OS, CUDA 11.6, Intel (R) Xeon (R) E-2244G CPU (4.80 GHz), Nvidia GeForce RTX 4070Ti GPU (12 GB), mainly based on Python 3.8.13 as the development language support. PyCharm 2021.3.1 IDE implements a parallel computing environment. We will display the running time of five methods on six protein–protein interaction network datasets (FAA4, Natoc_0297, 16 items, HOXA10, NNMT, and MBP) and the MNIST handwritten digit recognition dataset (only the trained part is selected), as shown in Figure 6.

From the results, the HDHPA method has an overall running time less than the other methods in all datasets, with a duration of less than 10 s. In the HOXA10 dataset, except for the HDHPA method, all other methods have a running time of more than 10 s, with a maximum running time of 18.4 s. The running time of all methods shows a consistent pattern. We can see that the speed of hypergraph-clustering methods can be ranked (from fastest to slowest) as HDHPA, kD, AD, kh, and Ah. We can see that using association rules to construct hypergraphs requires the iterative processing of complex and large-scale datasets in a non-parallel search regime, which leads to longer running time. Therefore, parallel search can significantly reduce the running time of processing large-scale data. By comparing only six PPI network datasets, it can see that HDHPA is more efficient than AD and Ah. Therefore, considering both running time and classification accuracy, the HDHPA is more competitive.

Based on the values of clustering evaluation indicators in the experimental results, the next step is to discuss clustering efficiency. From Figure 7, the clustering efficiency of the AD method will change significantly with the characteristics of the datasets, and the Ah method is generally lower than other methods in clustering evaluation indicators. The FM-index and ARI values of the HDHPA method are substantially higher than other methods, and the highest values can be close to 0.9. Therefore, the HDHPA method has lower time costs and more stable performance on different datasets. Because the construction and partition of the hypergraph occupy most of the running time, the improvement of the traditional Apriori algorithm makes the construction of the hypergraph a parallel calculation, saving a lot of time and achieving a better trade-off between effectiveness and efficiency.

5. Conclusions and Discussion

This paper mainly focuses on clustering multi-dimensional datasets using a constructed hypergraph model. We propose a hypergraph-clustering method called HDHPA (high-dimensional data clustering method based on hypergraph partitioning using improved Apriori algorithm). The method starts by reducing the dimensionality of high-dimensional data and then performs clustering based on the hypergraph model. During the construction of the hypergraph, based on the knowledge of hypergraph theory, the traditional associative Apriori algorithm was improved. The HDHPA method uses parallel computing to eliminate the limitations of the Apriori algorithm in the iterative search of candidate sets, algorithm efficiency, and other aspects. The HDHPA can also reasonably measure the similarity between data in high-dimensional space. Then, using the dense subgraph segmentation DSP algorithm to segment the hypergraph can better merge similar dense subgraphs, it can improve clustering accuracy and obtain the final clustering result.

Experiments were conducted on six protein–protein interaction (PPI) network datasets and the MNIST handwritten digit recognition dataset. The experimental results demonstrate that the HDHPA method outperforms traditional clustering algorithms and other hypergraph-partitioning algorithms, achieving efficient clustering results on datasets of different modalities. This indicates that the HDHPA method has broad applicability in various domains. Finally, the running time of all methods was compared, and the results show the superior performance of the HDHPA method in terms of time complexity. It is worth noting that the performance of our proposed method is related to the constructed hypergraph structure, so there are still many potential issues with the HDHPA method. Firstly, the method proposed in this paper has not yet comprehensively integrated more features of nodes and hyperedges into high-dimensional data to construct hypergraphs. Secondly, when improving the Apriori algorithm, how to reduce time complexity should be more comprehensive. Therefore, our future research directions mainly have two aspects: On the one hand, we will continue to try to conduct experiments on datasets in different fields, including high-dimensional network datasets such as social networks and brain networks. We will adjust the parameters of the method appropriately based on the characteristics of different modal datasets. On the other hand, combined with the latest intelligent clustering technology, we will further study how to shorten the running time mechanism of the method.