Analysis of Hypergraph Signals via High-Order Total Variation

Abstract

Beyond pairwise relationships, interactions among groups of agents do exist in many real-world applications, but they are difficult to capture by conventional graph models. Generalized from graphs, hypergraphs have been introduced to describe such high-order group interactions. Inspired by graph signal processing (GSP) theory, an existing hypergraph signal processing (HGSP) method presented a spectral analysis framework relying on the orthogonal CP decomposition of adjacency tensors. However, such decomposition may not exist even for supersymmetric tensors. In this paper, we propose a high-order total variation (HOTV) form of a hypergraph signal (HGS) as its smoothness measure, which is a hyperedge-wise measure aggregating all signal values in each hyperedge instead of a pairwise one in most existing work. Further, we propose an HGS analysis framework based on the Tucker decomposition of the hypergraph Laplacian induced by the aforementioned HOTV. We construct an orthonormal basis from the HOTV, by which a new spectral transformation of the HGS is introduced. Then, we design hypergraph filters in both vertex and spectral domains correspondingly. Finally, we illustrate the advantages of the proposed framework by applications in label learning.

1. Introduction

Graphs model and process structured data in irregular domains [1,2,3], which can model real-world agents and their pairwise interactions by vertices and edges. However, various real-world systems have interactions among multiple agents, beyond pairwise, in many fields such as social networks [4], computational chemistry [5], many-body physics [6,7], neuroscience [8,9], ecology [10,11,12,13], biology [14,15], etc. In e-commerce, several strangers may share similar shopping preferences and buy the same product online. In some ecosystems, multiple species may mutually compete for food and territory and affect each other. Those interactions are known as high-order interactions, which are about influence or similarity at the level of groups of agents [16,17]. A high-order interaction can not only correspond to a specific semantic attribute or behavior of agents, but it can also be an interplay among agents in a process or an overall similarity of agents. Researchers have been aware of the existence of high-order interactions for the past few decades.

A more general tool is needed to describe agents with such group interactions since a high-order interaction among a group of agents is not equivalent to the ensemble of pairwise interactions between any two members of the group in many scenarios. Hypergraphs are a candidate tool owing to the ability to connect multiple vertices simultaneously by one hyperedge [18]. Each hyperedge can connect a distinct number of vertices, which denotes a corresponding-order interaction of agents. Hypergraphs provide a natural and flexible way to model high-order interactions. A hypergraph signal (HGS) contains physical values defined on the vertices of a hypergraph.

Several tensor-based methods have been proposed to intuitively represent hypergraph topologies in algebra. Those methods take into consideration both uniform hypergraphs in which each hyperedge needs to connect the same number of vertices and general hypergraphs in which hyperedges can connect distinct numbers of vertices. Generalized from graph adjacency matrices, Cooper and Dutle [19] defined adjacency tensors for uniform hypergraphs. Further, Qi [20] presented the Laplacian and signless Laplacian of uniform hypergraphs. Extended from the above mathematical representations of uniform hypergraphs, both adjacency and Laplacian tensors of general hypergraphs were proposed in [21]. Additionally, some other Laplacians for uniform hypergraphs were presented in [22,23], combining hyperedge-wise and pairwise cutting cost functions, respectively. Ouvrard et al. [24] proposed e-adjacency tensors for general hypergraphs by first dividing a general hypergraph into multiple layers and then merging all layers by adding additional vertices.

1.1. Related Works

The hypergraph signal processing (HGSP) framework in [25] defined the hypergraph Fourier transform (HGFT) by the orthogonal CP decomposition [26] of hypergraph adjacency tensors. However, a large number of real supersymmetric tensors may not be superdiagonalized due to the possible large rank [27,28], let alone be orthogonally superdiagonalized. In such cases, the framework has to use an approximate decomposition instead, which may lead to imprecision in the subsequent HGFT and frequency interpretation.

The smoothness of the HGS draws researchers’ attention and there are many existing smoothness measures of the HGS. Utilizing high-order adjacency tensors, the total variation of the HGS defined in [25] is not a homogeneous polynomial. A large difference between the high-order shift result of the HGS over the hypergraph and the original 1st-order signal may also be caused by the different powers of the two signals rather than the unsmoothness. Based on the hypergraph cut, Hein et al. [29] defined a total variation combining the maximum pairwise difference of the HGS in each hyperedge. Existing smoothness measures over hypergraphs [30] were summarized or proposed as the combinations of pairwise smoothness measures in all hyperedges.

In addition to hypergraphs, there are other models proposed to capture high-order interactions among real-world agents. Graph approximation methods such as clique expansion [31] and star expansion [31,32] mostly project high-order interactions onto pairwise ones linearly and are able to utilize graphs and matrices to model high-order interactions [33]. Those methods will bring information loss of high-order interactions since the projections lead to the decrease of the interaction order and are irreversible.

Simplicial complexes are another tool for modeling high-order interactions in many scenarios. In [34,35], each simplex in a simplicial complex represents an interaction and has a signal defined on it. The framework processes signals defined on interactions and only takes into account the adjacency of simplices with the same dimension. Two k-dimension simplices are adjacent only if they are faces of the same $(k+1)$-dimension simplex or own a $(k-1)$-dimension simplex as their common face. For $k=0$, the simplices (vertices) have the smallest dimension, and therefore, their adjacency only depends on the one-dimensional simplices (edges). That is, interactions of vertices with orders greater than 2 are not taken into account while processing signals defined on vertices.

1.2. Main Works

In this paper, we model high-order interactions by a hypergraph and represent both the topology and the signal of an HGS by tensors. Unlike the total variation based on the adjacency tensor in [25] and some existing smoothness measures combining pairwise dissimilarities of signals [30], we propose a total variation of the HGS to measure the smoothness by taking into account smoothness measures over each hyperedge directly in a high-order perspective. Based on the proposed high-order total variation (HOTV) of the HGS, we obtain a hypergraph Laplacian tensor and construct an orthonormal basis of the HGS space to capture different spectral components of the HGS. We then propose an HGS analysis framework utilizing the basis in the Tucker decomposition form of the hypergraph Laplacian since a large number of real supersymmetric tensors do not have their orthogonal superdiagonalization forms. Our main contributions in this work are listed below:

• We propose an HOTV over hypergraphs, by which we obtain a hypergraph Laplacian and present an orthonormal basis reflecting distinct spectral information. The HOTV aggregates the HGS groupwise instead of pairwise, which describes the dissimilarity of the HGS over the topology in a more comprehensive way;
• We propose a novel signal transformation (a new HGFT) by the orthonormal basis which bridges the vertex domain and the spectral domain of an HGS. We then can process the HGS in the two domains, clearly provide spectral interpretations for all processing of the HGS and put forward a framework for the analysis and processing of the HGS;
• We present hypergraph filtering tasks in the two domains and discuss two specific forms of hypergraph filters, which do provide a new idea for the HGS filtering.

The rest of the paper is organized as follows. Section 2 introduces the HGS from the HOTV and the corresponding tensor-based representation. In Section 3, we construct a representative orthonormal basis, propose a novel HGFT and provide its spectral interpretation. In Section 4, hypergraph filtering and two specific forms of hypergraph filters are presented. We then provide an application of the proposed framework and show its advantage by some experimental results in Section 5, before finally concluding the paper in Section 6. Additionally, we first list the notations used in the paper in

Table 1. Notations with descriptions.

Notation	Description
$\mathcal{H}=(\mathcal{V},\mathcal{E},\mathbf{W})$	a c-uniform weighted undirected hypergraph
${\mathbf{L}}_{\left(c\right)}$	the Laplacian tensor of $\mathcal{H}$
$\mathit{s}$	an HGS
${\mathit{s}}_{\left(M\right)}$	an Mth-order HGS
${\mathcal{S}}_{\left(M\right)}$	the Mth-order HGS set
$\widehat{\mathit{s}}$	a spectral-domain HGS
$\mathbf{F}$	a vertex-domain hypergraph filter
$\widehat{\mathbf{F}}$	a spectral-domain hypergraph filter
⊗	the Kronecker product
⊠	the outer product
${\times }_n$	the n-mode product
⊙	the Khatri–Rao product
$\left[n\right]$	$\{1,\cdots ,n\}$
$\mathrm{mat}(·)$	tensor matricization
$\mathrm{vec}(·)$	tensor vectorization
$\mathrm{ten}(·)$	tensorization of a matrix or a vector
$\mathrm{ROA}(·)$	the rank-one approximation of any form of a tensor

below.

2. Hypergraph Signals

A weighted undirected hypergraph $\mathcal{H}=(\mathcal{V},\mathcal{E},\mathbf{W})$ consists of a vertex set $\mathcal{V}$, a hyperedge set $\mathcal{E}$ and a diagonal hyperedge weight matrix $\mathbf{W}\in {\mathbb{R}}_{\ge 0}^{\left|\mathcal{E}\right|\times \left|\mathcal{E}\right|}$. Each hyperedge $\mathit{e}\in \mathcal{E}\mathcal{E}$ is a nonempty subset of the vertex set $\mathcal{V}$. The cardinality of hyperedge $\mathit{e}$ denotes the number of vertices $\mathit{e}$ connects. If all hyperedges have the same cardinality c, the hypergraph is c-uniform. A real-valued HGS $s:\mathcal{V}\mapsto \mathbb{R}$ defined on the vertices of hypergraph $\mathcal{H}$ can be represented by a vector $\mathit{s}\in {\mathbb{R}}^{\left|\mathcal{V}\right|}$, whose ith entry $s_i$ represents the signal value at the ith vertex in $\mathcal{V}$.

Example 1.

The 4-uniform weighted undirected hypergraph in Figure 1 consists of nine vertices ${\left\{v_i\right\}}_{i=1}^9$ and three 4-cardinality hyperedges ${\left\{{\mathit{e}}_i\right\}}_{i=1}^3$ with weights equal to $0.8$, $0.2$ and 1, respectively. The hyperedge ${\mathit{e}}_1$ connects four vertices, i.e., $v_1$, $v_2$, $v_5$ and $v_6$.

In our work, we consider the weighted undirected hypergraph $\mathcal{H}=(\mathcal{V},\mathcal{E},\mathbf{W})$ as a c-uniform hypergraph with a signal $\mathit{s}\in {\mathbb{R}}^{\left|\mathcal{V}\right|}$ where c is an even number. If c is odd, we preprocess all hyperedges by adding an auxiliary vertex to each of them to make their cardinalities even in our previous work [36]. Specifically, we denote the preprocessed hypergraph by ${\mathcal{H}}^{\prime }=({\mathcal{V}}^{\prime },{\mathcal{E}}^{\prime },\mathbf{W})$ where the weight matrix remains unchanged. Note that each auxiliary vertex only appears in one hyperedge. All $\left|\mathcal{E}\right|$ auxiliary vertices corresponding to each hyperedge are denoted by $v_i$ for $i=\left|\mathcal{V}\right|+1,\cdots ,\left|\mathcal{V}\right|+\left|\mathcal{E}\right|$, and the new vertex set ${\mathcal{V}}^{\prime }=\mathcal{V}\cup {\left\{v_i\right\}}_{i=\left|\mathcal{V}\right|+1}^{\left|\mathcal{V}\right|+\left|\mathcal{E}\right|}$ contains both original and auxiliary vertices. Each auxiliary vertex $v_{\left|\mathcal{V}\right|+i}$ is added into the original hyperedge ${\mathit{e}}_i$ to obtain the new hyperedge ${\mathit{e}}_i^{\prime }$ represented by ${\mathit{e}}_i^{\prime }={\mathit{e}}_i\cup \left\{v_{\left|\mathcal{V}\right|+i}\right\}$. The signal at each auxiliary vertex $v_{\left|\mathcal{V}\right|+i}$ in the new hyperedge ${\mathit{e}}_i^{\prime }$ is the arithmetic mean of signals on all vertices in the original hyperedge ${\mathit{e}}_i$ formulated as $s_{\mathrm{aux}}=\frac{1}{c}{\sum }_{v_i\in \mathit{e}}{s}_i$. Thus, the preprocessed HGS ${\mathit{s}}^{\prime }\in {\mathbb{R}}^{|{\mathcal{V}}^{\prime }|}$ can be obtained by ${\mathit{s}}^{\prime }={[{\mathbf{I}}_{\left|\mathcal{V}\right|},\frac{1}{c}\mathbf{H}]}^T\mathit{s}$, in which matrices ${\mathbf{I}}_{\left|\mathcal{V}\right|}\in {\mathbb{R}}^{\left|\mathcal{V}\right|\times \left|\mathcal{V}\right|}$ and $\mathbf{H}\in {\{0,1\}}^{\left|\mathcal{V}\right|\times \left|\mathcal{E}\right|}$ are the identity matrix and the incidence matrix of the original hypergraph $\mathcal{H}$, respectively. We then can obtain a $(c+1)$-uniform hypergraph with even-cardinality hyperedges.

In the preprocessing, the introduction of the auxiliary vertices slightly changes the topology of the hypergraph. Specifically, it makes minor changes to the topology of each hyperedge locally, which still retains the global neighboring relationships of all $\left|\mathcal{V}\right|$ original vertices.

Example 2.

For simplicity, we consider a 3-cardinality hyperedge $\mathit{e}=\{v_1,v_2,v_3\}$ with weight $w_{\mathit{e}}=1$ in Figure 2. We preprocess it by adding an auxiliary vertex into $\mathit{e}$ and obtain a new hyperedge ${\mathit{e}}^{\prime }=\mathit{e}\cup \left\{v_{\mathrm{aux}}\right\}$. The weight of ${\mathit{e}}^{\prime }$ remains 1, the vertex set becomes $\mathcal{V}\cup \left\{v_{\mathrm{aux}}\right\}$, and the signal at the auxiliary vertex is the arithmetic mean of signals on vertices ${\left\{v_i\right\}}_{i=1}^3$ denoted by $s_{\mathrm{aux}}=\frac{1}{3}{\sum }_{i=1}^3{s}_i$.

For a general hypergraph consisting of hyperedges with different cardinalities, we can preprocess all odd-cardinality hyperedges first. We then can divide the preprocessed hypergraph into a set of uniform partial hypergraphs with even cardinalities.

2.1. Total Variation of Hypergraph Signals

Inspired by the total variation of the graph signal, we generalize its notion to the total variation of the HGS to measure and describe the smoothness of the HGS over the hypergraph topology. Utilizing high-order adjacency tensors, the total variation of the HGS defined in [25] is not a homogeneous polynomial. Existing smoothness measures over hypergraphs [30] are combinations of pairwise smoothness measures in all hyperedges. Unlike those methods, we measure the smoothness of the HGS by a high-order homogeneous polynomial combining hyperedge-wise dissimilarities of signals.

We extend and define the HOTV of the HGS $\mathit{s}$ over the c-uniform hypergraph $\mathcal{H}$ taking the form of

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \mathrm{TV}\left(\mathit{s}\right)≔\frac{1}{n\left(c\right)}\sum _{\mathit{e}\in \mathcal{E}}{w}_{\mathit{e}}\sum _{v_i\in \mathit{e}}{(s_i-{\overline{s}}_{\mathit{e}})}^c,\end{array}\end{array}$$

(1)

where $n\left(c\right)=\frac{{(c-1)}^c+c-1}{c^c}$ normalizes the vertex degree for each hyperedge, $w_{\mathit{e}}$ denotes the weight of the hyperedge $\mathit{e}$, and ${\overline{s}}_{\mathit{e}}\frac{1}{c}{\sum }_{v_i\in \mathit{e}}{s}_i$ is the arithmetic mean of signals at each vertex in hyperedge $\mathit{e}$. ${\overline{s}}_{\mathit{e}}$ can be viewed as the equally weighted linear aggregation of signals on all vertices in $\mathit{e}$. Each vertex in $\mathit{e}$ is treated equivalently by the hyperedge and makes the same contribution to the HOTV over $\mathit{e}$. The HOTV considers the cth-order hyperedge-wise differences among signals rather than pairwise differences between signals. As a smoothness measure of the HGS $\mathit{s}$ over the c-uniform hypergraph $\mathcal{H}$ where c is even, the HOTV is nonnegative for $\forall \mathit{s}\in {\mathbb{R}}^{\left|\mathcal{V}\right|}$. The HOTV equals zero if and only if vertices in each connected component of the hypergraph $\mathcal{H}$ take the same signal value. The HOTV will be small if the HGS obtains similar values on neighboring vertices, while the HOTV will be large if the HGS varies greatly over the topology.

When $c=2$, the hypergraph $\mathcal{H}$ becomes a graph and the total variation (1) has the same form as the quadratic form of graph Laplacians $\mathbf{L}$ formulated as

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \mathrm{TV}\left(\mathit{s}\right)=2\sum _{\mathit{e}=\{v_i,v_j\}\in \mathcal{E}}{w}_{\mathit{e}}({(s_i-{\overline{s}}_{\mathit{e}})}^2+{(s_j-{\overline{s}}_{\mathit{e}})}^2)=\sum _{\mathit{e}=\{v_i,v_j\}\in \mathcal{E}}{w}_{\mathit{e}}{(s_i-s_j)}^2={\mathit{s}}^T\mathbf{L}\mathit{s}.\end{array}\end{array}$$

(2)

Therefore, the total variation of graph signals can be considered as a special case of the proposed generalized HOTV.

Example 3.

The HOTV over the 4-uniform weighted hypergraph in Figure 1 is a quartic homogeneous polynomial given by

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \begin{array}{cc}\hfill \mathrm{TV}\left(\mathit{s}\right)& =w_{{\mathit{e}}_1}\sum _{v_i\in {\mathit{e}}_1}{(s_i-{\overline{s}}_{{\mathit{e}}_1})}^4+w_{{\mathit{e}}_2}\sum _{v_i\in {\mathit{e}}_2}{(s_i-{\overline{s}}_{{\mathit{e}}_2})}^4+w_{{\mathit{e}}_3}\sum _{v_i\in {\mathit{e}}_3}{(s_i-{\overline{s}}_{{\mathit{e}}_3})}^4\hfill \\ & =0.8\sum _{v_i\in \{v_1,v_2,v_5,v_6\}}{(s_i-\frac{s_1+s_2+s_5+s_6}{4})}^4+\hfill \\ & 0.2\sum _{v_i\in \{v_2,v_3,v_4,v_5\}}{(s_i-\frac{s_2+s_3+s_4+s_5}{4})}^4+\sum _{v_i\in \{v_3,v_7,v_8,v_9\}}{(s_i-\frac{s_3+s_7+s_8+s_9}{4})}^4,\hfill \end{array}\end{array}\end{array}$$

which can be seen as hyperedge-wise measuring the high-order dissimilarity among signals. When the HGS $\mathit{s}$ obtains the same value at each vertex, each hyperedge-wise term in $\mathrm{TV}\left(\mathit{s}\right)$ and thus the whole $\mathrm{TV}\left(\mathit{s}\right)$ all equal zero.

For a preprocessed general hypergraph consisting of hyperedges with different even cardinalities, the HOTV (1) is no longer a homogeneous polynomial. Instead, it is the sum of the total variations of all the uniform partial hypergraphs formulated as

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \mathrm{TV}\left(\mathit{s}\right)=\sum _{c\in \mathcal{C}}{\mathrm{TV}}_{\left(c\right)}\left(\mathit{s}\right),\end{array}\end{array}$$

(3)

where $\mathcal{C}$ is the cardinality set. The total variation of each uniform partial hypergraph is still a homogeneous polynomial. It is worth mentioning that the preprocessing of odd-cardinality partial hypergraphs ensures the nonnegative property of the total variation. Moreover, the hypergraph preprocessing does not change the intrinsic essence of the total variation, since the difference term corresponding to each auxiliary vertex equals zero and thus makes no contribution to the total variation. For clarity, we provide an example here to show how the minor changes of the topology slightly affect the difference polynomial and allow it to meet the definition of the total variation.

Example 4.

For simplicity, we continue considering the 3-cardinality hyperedge $\mathit{e}=\{v_1,v_2,v_3\}$ with weight 1 in Figure 2. The total variation over ${\mathit{e}}^{\prime }$ is given by

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \mathrm{TV}\left(\mathit{s}\right)=\frac{1}{n\left(4\right)}\sum _{i=1}^3{(s_i-{\overline{s}}_{{\mathit{e}}^{\prime }})}^4+{({\overline{s}}_e-{\overline{s}}_{{\mathit{e}}^{\prime }})}^4=\frac{1}{n\left(4\right)}\sum _{i=1}^3{(s_i-{\overline{s}}_{{\mathit{e}}^{\prime }})}^4.\end{array}\end{array}$$

The difference term of the auxiliary vertex equals zero since ${\overline{s}}_e={\overline{s}}_{{\mathit{e}}^{\prime }}=\frac{1}{3}{\sum }_{i=1}^3{s}_i$. Obviously, the total variation over the new hyperedge ${\mathit{e}}^{\prime }$ still only considers the difference between the signal at each original vertex and their arithmetic mean. The preprocessing of $\mathit{e}$ changes only the order of the polynomial from 3 to 4, since $\mathrm{TV}\left(\mathit{s}\right)$ is defined only when c is even.

2.2. Tensor-Based Representations

As multidimensional arrays, tensors are a natural tool to mathematically represent hypergraph topologies and the corresponding high-order signals thanks to the flexibility of the tensor order. As a bijection between the index sets of two tensors that have the same size and different shapes, tensor reshaping such as tensor matricization and tensor vectorization can be utilized to equivalently represent any high-order HGS by various forms, such as matrix and vector forms. We will use operators $\mathrm{mat}(·)$, $\mathrm{vec}(·)$ to represent the two kinds of tensor reshaping listed above and the operator $\mathrm{ten}(·)$ for the recovery from other forms to the tensor form.

2.2.1. Representations of Topologies

As mentioned in Section 2.1, the HOTV (1) measures the dissimilarity of the HGS. It can be rewritten as

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \begin{array}{cc}\hfill \mathrm{TV}\left(\mathit{s}\right)& =\frac{1}{n\left(c\right)}\sum _{\mathit{e}\in \mathcal{E}}{w}_{\mathit{e}}\sum _{v_i\in \mathit{e}}{(\frac{c-1}{c}{s}_i-\frac{1}{c}\sum _{v_j\in \mathit{e}\setminus \left\{v_i\right\}}{s}_j)}^c\hfill \\ & =\frac{1}{n\left(c\right)c^c}\sum _{\mathit{e}\in \mathcal{E}}{w}_{\mathit{e}}\sum _{i=1}^c{({(c{\mathit{e}}_i-\mathbf{1})}^T{\mathrm{\Phi }}_{\mathit{e}}\mathit{s})}^c\hfill \\ & =\frac{1}{n\left(c\right)c^c}\sum _{\mathit{e}\in \mathcal{E}}{w}_{\mathit{e}}\sum _{i=1}^c{\left({\mathrm{\Phi }}_{\mathit{e}}^T(c{\mathit{e}}_i-\mathbf{1})\right)}^{⊠c}{\times }_1{\mathit{s}}^T{\times }_2\cdots {\times }_c{\mathit{s}}^T\hfill \\ & =\frac{1}{n\left(c\right)c^c}\sum _{\mathit{e}\in \mathcal{E}}{w}_{\mathit{e}}\sum _{i=1}^c{\left({\mathrm{\Phi }}_{\mathit{e}}^T(c{\mathit{e}}_i-\mathbf{1})\right)}^{⊠c}{\mathit{s}}^c,\hfill \end{array}\end{array}\end{array}$$

(4)

where the binary matrix ${\mathrm{\Phi }}_{\mathit{e}}\in {\mathbb{R}}^{c\times \left|\mathcal{V}\right|}$ samples all vertices in $\mathit{e}$ from the vertex set $\mathcal{V}$, the vector ${\mathit{e}}_i$ denotes the ith column of the $c\times c$ identity matrix ${\mathbf{I}}_c$, the vector $\mathbf{1}$ is a c-dimension all-ones vector, the operator ⊠ is the outer product of two vectors (or matrices), and the operator ${\times }_n$ denotes the n-mode product of a tensor and a matrix. From (4), we can accordingly obtain a cth-order $\left|\mathcal{V}\right|$-dimension hypergraph Laplacian tensor ${\mathbf{L}}_{\left(c\right)}$ containing all information of the topology as an HGS difference operator formulated as

$$\begin{array}{c}\hfill \begin{array}{c}\hfill {\mathbf{L}}_{\left(c\right)}=\frac{1}{n\left(c\right)c^c}\sum _{\mathit{e}\in \mathcal{E}}{w}_{\mathit{e}}\sum _{i=1}^c{\left({\mathrm{\Phi }}_{\mathit{e}}^T(c{\mathit{e}}_i-\mathbf{1})\right)}^{⊠c},\end{array}\end{array}$$

(5)

which is found to coincide with the Laplacian tensor defined in [22]. The HOTV can then be represented by ${\mathbf{L}}_{\left(c\right)}$ as

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \mathrm{TV}\left(\mathit{s}\right)={\mathbf{L}}_{\left(c\right)}{\mathit{s}}^c\end{array}\end{array}$$

(6)

for a c-uniform hypergraph, and

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \mathrm{TV}\left(\mathit{s}\right)=\sum _{c\in \mathcal{C}}{\mathbf{L}}_{\left(c\right)}{\mathit{s}}^c\end{array}\end{array}$$

(7)

for a general hypergraph. ${\mathbf{L}}_{\left(c\right)}$ in (7) denotes the cth-order Laplacian tensor of the c-uniform partial hypergraph.

It is obvious from the definition (5) that the hypergraph Laplacian is supersymmetric [27], which means that its entries are invariant under any permutation of its indices. Moreover, according to both the nonnegative property of the HOTV and the hypergraph Laplacian form, we obtain Proposition 1 [22]. The definition of positive semidefinite tensors is presented below as well. In general, extended from the graph Laplacian, the even-order hypergraph Laplacian is a high-order difference operator which is supersymmetric and positive semidefinite as well.

Definition 1

(Positive semidefinite tensor). A real mth-order n-dimension supersymmetric tensor $\mathbf{T}=\left(t_{i_1,\cdots ,i_m}\right)$, $1\le i_1,\cdots ,i_m\le n$ is positive semidefinite if for $\forall \mathit{x}\in {\mathbb{R}}^n$, the homogeneous polynomial

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \mathbf{T}{\mathit{x}}^m=\mathbf{T}{\times }_1{\mathit{x}}^T{\times }_2\cdots {\times }_m{\mathit{x}}^T=\sum _{i_1,\cdots i_m=1}^n{t}_{i_1,\cdots ,i_m}{\mathit{x}}_{i_1}\cdots {\mathit{x}}_{i_m}\ge 0.\end{array}\end{array}$$

(8)

Proposition 1.

Even-order hypergraph Laplacian tensors (5) are positive semidefinite.

For a preprocessed general hypergraph, we can obtain an even-order Laplacian tensor set according to the corresponding even-cardinality uniform partial hypergraph set. We can subsequently process the Laplacian tensor of each order accordingly.

2.2.2. Representations of Signals

For a real-valued HGS $\mathit{s}\in {\mathbb{R}}^{\left|\mathcal{V}\right|}$, we introduce nonlinearity and define a high-order signal function $s_{\left(M\right)}:\underset{M}{\underbrace{\mathcal{V}\times \cdots \times \mathcal{V}}}\mapsto \mathbb{R}$ to obtain high-order signal values among vertices, where $M=c/2$ and the operator × is the Cartesian product of two sets. The high-order HGS ${\mathit{s}}_{\left(M\right)}\in {\mathcal{S}}_{\left(M\right)}\subseteq {\mathbb{R}}^{{\left|\mathcal{V}\right|}^M}$ is obtained by a nonlinear function of the original signal $\mathit{s}$ formulated as

$$\begin{array}{c}\hfill \begin{array}{c}\hfill {\mathit{s}}_{\left(M\right)}≔\mathrm{vec}\left({\mathit{s}}^{⊠M}\right)={\mathit{s}}^{\otimes M},\end{array}\end{array}$$

(9)

where ${\mathcal{S}}_{\left(M\right)}$ is a subset of ${\mathbb{R}}^{{\left|\mathcal{V}\right|}^M}$ formed by all values of ${\mathit{s}}_{\left(M\right)}$ for $\forall \mathit{s}\in {\mathbb{R}}^{\left|\mathcal{V}\right|}$, and the operator ⊗ denotes the Kronecker product of two vectors (or matrices). The supersymmetric rank-one [27,37] tensor ${\mathit{s}}^{⊠M}=\mathrm{ten}\left({\mathit{s}}_{\left(M\right)}\right)$ is the tensor form of the high-order HGS. We denote the set $\{1,\cdots ,|\mathcal{V}\left|\right\}$ by $\left[\right|\mathcal{V}\left|\right]$. For $\forall i_1,i_2,\cdots ,i_M\in \left[\right|\mathcal{V}\left|\right]$, the corresponding entry of ${\mathit{s}}_{\left(M\right)}$ is

$$\begin{array}{c}\hfill \begin{array}{c}\hfill {\left({\mathit{s}}_{\left(M\right)}\right)}_{1+{\sum }_{m=1}^M(i_m-1){\left|\mathcal{V}\right|}^{M-m}}=s_{i_1}{s}_{i_2}\cdots s_{i_M}.\end{array}\end{array}$$

(10)

Remark (Sign ambiguity): There exists a problem introduced by the Mth-order signals when $M=c/2$ is an even number. We cannot judge the sign of an HGS from the Mth-order form of the HGS. Instead, we can only determine the relative sign, namely, whether signals at any pair of vertices have the same or different signs. In this case, we may obtain the original HGS or a signal with the opposite sign from its high-order form. In semi-supervised or supervised learning tasks, we can determine the sign of an HGS by both the relative signs and signs of observations. For unsupervised learning tasks such as vertex clustering, it is enough to obtain distances or similarities of vertices from topologies that are irrelevant to signals. Therefore, although there may be no mapping from high-order signals to the original 1st-order ones due to the sign ambiguity, it does not matter in many practical applications. While degenerating into the graph case, namely, $M=1$, the high-order signal coincides with the original one and thus the above problem does not exist.

3. Hypergraph Fourier Transform

3.1. Construction of an Orthonormal Basis

We assume that the HGS is correlated to the topology and evolves smoothly over the topology since we construct the hypergraph based on high-order interactions among vertices. We need a basis for the HGS to capture the main features of the HGS and find relatively smooth representations of the HGS in a low-dimension vector space. Utilizing the high-order difference property of the HOTV (4), we can obtain the basis by decomposing the hypergraph Laplacian tensor. It is reasonable to treat each mode equivalently and obtain the same basis in the tensor decomposition since the tensor ${\mathbf{L}}_{\left(c\right)}$ is supersymmetric. We require the basis vectors to be orthogonal since we hope for the HGS with distinct features to be irrelevant and to share no signal components. We then represent the orthonormal basis for ${\mathbb{R}}^{\left|\mathcal{V}\right|}$, where the HGS is located, by a matrix $\mathbf{U}\in {\mathbb{R}}^{\left|\mathcal{V}\right|\times \left|\mathcal{V}\right|}$. Accordingly, the tensor decomposition formulated as

$$\begin{array}{c}\hfill \begin{array}{c}\hfill {\mathbf{L}}_{\left(c\right)}={\mathbf{G}}_{\left(c\right)}{\times }_1\mathbf{U}{\times }_2\cdots {\times }_c\mathbf{U}\end{array}\end{array}$$

(11)

conforms to the Tucker decomposition form [27] which allows the core tensor ${\mathbf{G}}_{\left(c\right)}$ not to be superdiagonal and assumes the basis $\mathbf{U}$ usually to be orthogonal. The decomposition (11) of ${\mathbf{L}}_{\left(c\right)}$ with the basis $\mathbf{U}$ always exists owing to the existence and the non-uniqueness of the Tucker decomposition [27].

${\mathbf{U}}^{\otimes M}$ is an orthonormal matrix since we require $\mathbf{U}$ to be orthonormal. Considering high-order interactions among vertices, we can obtain the orthonormal basis ${\mathbf{U}}^{\otimes M}$ for the high-order HGS by calculating the ith basis vector ${\mathit{u}}_i$ of $\mathbf{U}$ for $i=1,\cdots ,\left|\mathcal{V}\right|$ in turn, which solves a functional minimization problem formulated as

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \begin{array}{cc}\hfill {\mathit{u}}_i^{\otimes M}=& \underset{{\mathit{s}}_{\left(M\right)}\in {\mathcal{S}}_{\left(M\right)}}{argmin}\mathrm{TV}\left({\mathit{s}}_{\left(M\right)}\right)\hfill \\ \hfill & \begin{array}{ccc}\hfill \mathrm{s}.\mathrm{t}.& \hfill {\left({\mathbf{U}}_{i-1}^{\odot M}\right)}^T{\mathit{s}}_{\left(M\right)}& \hfill =\mathbf{0}\\ \hfill & \hfill {\mathit{s}}_{\left(M\right)}^T{\mathit{s}}_{\left(M\right)}& \hfill =1,\end{array}\hfill \end{array}\end{array}\end{array}$$

(12)

where the operator ⊙ denotes the Khatri–Rao product of two matrices with the same number of columns and the matrix ${\mathbf{U}}_{i-1}^{\odot M}=[{\mathit{u}}_1^{\otimes M},\cdots ,{\mathit{u}}_{i-1}^{\otimes M}]\in {\mathbb{R}}^{{\left|\mathcal{V}\right|}^M\times (i-1)}$ is only defined for $i\ge 2$. The first constraint in (12) containing $i-1$ equations denotes the orthogonal constraints of the basis vectors and only exists when $i\ge 2$. The second constraint in (12) requires the high-order signal to be normalized. By solving (12) iteratively, we can obtain a set of orthonormal high-order signals in a sequence that keeps the HOTV nondecreasing. In other words, for the ith problem, we obtain the smoothest signal as the basis vector which is orthogonal to the first $i-1$ obtained ones.

The constraint ${\mathit{s}}_{\left(M\right)}^T{\mathit{s}}_{\left(M\right)}=1$ is equivalent to ${\mathit{s}}^T\mathit{s}=1$, since the real-valued signal satisfies ${\mathit{s}}_{\left(M\right)}^T{\mathit{s}}_{\left(M\right)}={\left({\mathit{s}}^T\mathit{s}\right)}^{\otimes M}={\left({\mathit{s}}^T\mathit{s}\right)}^M$ according to (9) and the inner product ${\mathit{s}}^T\mathit{s}$ is nonnegative. Thus, we can consider the 1st-order form of HGS $\mathit{s}$ directly and rewrite (12) as

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \begin{array}{cc}\hfill {\mathit{u}}_i=& \underset{\mathit{s}\in {\mathbb{R}}^{\left|\mathcal{V}\right|}}{argmin}\mathrm{TV}\left(\mathit{s}\right)\hfill \\ \hfill & \begin{array}{ccc}\hfill \mathrm{s}.\mathrm{t}.& \hfill {\mathbf{U}}_{i-1}^T\mathit{s}& \hfill =\mathbf{0}\\ \hfill & \hfill {\mathit{s}}^T\mathit{s}& \hfill =1,\end{array}\hfill \end{array}\end{array}\end{array}$$

(13)

where ${\mathbf{U}}_{i-1}=[{\mathit{u}}_1,\cdots ,{\mathit{u}}_{i-1}]\in {\mathbb{R}}^{\left|\mathcal{V}\right|\times (i-1)}$ for $i\ge 2$. We provide a solution by the method of Lagrange multipliers and gradient descent method in Appendix A, since the problem (13) involves tensor operations such as tensor products and partial derivatives. As is also discussed in Appendix A, there may be nonzero off-diagonal entries in the core tensor ${\mathbf{G}}_{\left(c\right)}$ for the Tucker decomposition, specifically with the basis $\mathbf{U}$.

Example 5.

Three basis vectors ${\mathit{u}}_1$, ${\mathit{u}}_2$ and ${\mathit{u}}_8$ of the example hypergraph $\mathcal{H}$ in Figure 1 are presented in Figure 3. As the first basis vector, namely the smoothest one, ${\mathit{u}}_1$ is a constant HGS over the topology with a zero total variation. The basis vector ${\mathit{u}}_2$ seems to provide an informative vertex clustering method corresponding to a specific smoothness of the HGS. Note that basis vectors associated with small total variations vary slowly over the topology, while basis vectors associated with larger total variations oscillate more rapidly and disorderly and tend to have more dissimilar values on vertices connected by large-weight hyperedges.

When $c=2$, problems (12) and (13) degenerate to the graph case and become exactly the same. By the Courant–Fischer theorem, the ith smallest eigenvalue ${\lambda }_i$ of a graph Laplacian matrix is the value of $\mathrm{TV}\left({\mathit{u}}_i\right)$, and the corresponding eigenvector ${\mathit{u}}_i$ can be obtained by either (12) or (13). In either of the ways above, we can obtain a matrix $\mathbf{U}=[{\mathit{u}}_1,\cdots ,{\mathit{u}}_{\left|\mathcal{V}\right|}]$ which contains the basis of the graph Fourier transform (GFT) as its columns. Therefore, one can view the proposed orthonormal basis as a generalization of the GFT basis in the hypergraph case.

3.2. Hypergraph Fourier Transform

Given the orthonormal basis $\mathbf{U}$ generalized from the GFT basis, we define a novel HGFT by projecting signals to the orthonormal basis in order to process the high-order HGS in both vertex and spectral domains. The HGFT transforms the high-order vertex-domain HGS ${\mathit{s}}_{\left(M\right)}$ into the spectral-domain signal ${\widehat{\mathit{s}}}_{\left(M\right)}$, which can be formulated in both vector and tensor forms as

$$\begin{array}{ccccc}\hfill \left(\mathbf{vector}\mathbf{form}\right)& & \hfill {\widehat{\mathit{s}}}_{\left(M\right)}& {\left({\mathbf{U}}^{\otimes M}\right)}^T{\mathit{s}}_{\left(M\right)}={\left({\mathbf{U}}^T\mathit{s}\right)}^{\otimes M}\hfill & \hfill \end{array}$$

(14)

$$\begin{array}{ccccc}\hfill \left(\mathbf{tensor}\mathbf{form}\right)& & \hfill \mathrm{ten}\left({\widehat{\mathit{s}}}_{\left(M\right)}\right)& =\mathrm{ten}\left({\mathit{s}}_{\left(M\right)}\right){\times }_1{\mathbf{U}}^T{\times }_2\cdots {\times }_M{\mathbf{U}}^T={\left({\mathbf{U}}^T\mathit{s}\right)}^{⊠M}.\hfill & \hfill \end{array}$$

(15)

The inverse HGFT (IHGFT) is accordingly defined as

$$\begin{array}{ccccc}\hfill \left(\mathbf{vector}\mathbf{form}\right)& & \hfill {\mathit{s}}_{\left(M\right)}& ={\mathbf{U}}^{\otimes M}{\widehat{\mathit{s}}}_{\left(M\right)}={\left(\mathbf{U}\widehat{\mathit{s}}\right)}^{\otimes M}\hfill & \hfill \end{array}$$

(16)

$$\begin{array}{ccccc}\hfill \left(\mathbf{tensor}\mathbf{form}\right)& & \hfill \mathrm{ten}\left({\mathit{s}}_{\left(M\right)}\right)& =\mathrm{ten}\left({\widehat{\mathit{s}}}_{\left(M\right)}\right){\times }_1\mathbf{U}{\times }_2\cdots {\times }_M\mathbf{U}={\left(\mathbf{U}\widehat{\mathit{s}}\right)}^{⊠M},\hfill & \hfill \end{array}$$

(17)

which transforms a high-order spectral-domain signal ${\widehat{\mathit{s}}}_{\left(M\right)}$ into its vertex-domain form ${\mathit{s}}_{\left(M\right)}$. When $M=1$, all bases and signals in both the HGFT and the IHGFT are the same as those in the graph case.

Obviously, only orthogonal transformation is involved in either the HGFT or the IHGFT since ${\mathbf{U}}^{\otimes M}$ is an orthonormal matrix. Therefore, signal transformations in both directions satisfy the conservation of energy.

To reduce the computational cost, we can calculate their equivalent forms (14) and (16) by the last step of each formula since (15) and (17) are all supersymmetric and rank-one. Moreover, both the HGFT (14) and the IHGFT (16) can degenerate into the transformation and its inverse of the original HGS given by

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \widehat{\mathit{s}}={\mathbf{U}}^T\mathit{s}\end{array}\end{array}$$

(18)

and

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \mathit{s}=\mathbf{U}\widehat{\mathit{s}}\end{array}\end{array}$$

(19)

in some reducible cases of the follow-up work, since the sign ambiguity brought by high-order signals does not matter, as we have discussed in Section 2.2.

3.3. Spectral Form of Total Variation

After the construction of the orthomormal basis and the definition of the HGFT, the HOTV of the HGS in the spectral domain can be written as

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \begin{array}{cc}\hfill \mathrm{TV}\left(\mathit{s}\right)& ={\mathbf{L}}_{\left(c\right)}{\times }_1{\mathit{s}}^T{\times }_2\cdots {\times }_c{\mathit{s}}^T\hfill \\ & =({\mathbf{G}}_{\left(c\right)}{\times }_1\mathbf{U}{\times }_2\cdots {\times }_c\mathbf{U}){\times }_1{\mathit{s}}^T{\times }_2\cdots {\times }_c{\mathit{s}}^T\hfill \\ & ={\mathbf{G}}_{\left(c\right)}{\times }_1\left({\mathit{s}}^T\mathbf{U}\right){\times }_2\cdots {\times }_c\left({\mathit{s}}^T\mathbf{U}\right)\hfill \\ & ={\mathbf{G}}_{\left(c\right)}{\times }_1{\widehat{\mathit{s}}}^T{\times }_2\cdots {\times }_c{\widehat{\mathit{s}}}^T={\mathbf{G}}_{\left(c\right)}{\widehat{\mathit{s}}}^c\hfill \\ & =\mathrm{TV}\left(\widehat{\mathit{s}}\right)\hfill \end{array}\end{array}\end{array}$$

(20)

according to (4) and (11). For $1\le i\le \left|\mathcal{V}\right|$, the ith diagonal entry of ${\mathbf{G}}_{\left(c\right)}$ denotes the HOTV of the ith basis vector of the HGS. Off-diagonal entries of ${\mathbf{G}}_{\left(c\right)}$ correspond to invalid signal components since they do not belong to ${\mathcal{S}}_{\left(M\right)}$. As is mentioned above, since we introduce nonlinearity by defining and processing the high-order HGS, the subset ${\mathcal{S}}_{\left(M\right)}$ is not a subspace of ${\mathbb{R}}^{{\left|\mathcal{V}\right|}^M}$. Therefore, those components are necessary for representing any valid HGS linearly even though they are invalid themselves.

In Section 3.2, we defined the spectral form of an HGS $\mathit{s}$ as $\widehat{\mathit{s}}$. We now analyze the spectral property of the HGS reflected by $\widehat{\mathit{s}}$. The ith entry of $\widehat{\mathit{s}}$ is the coefficient of projecting $\mathit{s}$ to the ith smoothest basis vector ${\mathit{u}}_i$. Therefore, entries of $\widehat{\mathit{s}}$ reflect the spectral distribution of the HGS from low-frequency to high-frequency.

Example 6.

The total variation of an $N\times N$ image $\mathit{X}$ under circular boundary conditions [38] can be viewed as the total variation of a 2-dimensional signal $\mathrm{vec}\left(\mathit{X}\right)\in {\mathbb{R}}^{N^2}$ formulated as

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \mathrm{TV}\left(\mathit{X}\right)={\mathrm{vec}\left(\mathit{X}\right)}^T\mathbf{L}\mathrm{vec}\left(\mathit{X}\right),\end{array}\end{array}$$

(21)

where $\mathbf{L}\in {\mathbb{R}}^{N^2\times N^2}$ is the Laplacian matrix of the Cartesian product of two identical N-vertex unweighted loops. The Laplacian matrix ${\mathbf{L}}_N$ of such N-vertex unweighted loop can always be diagonalized by the unitary discrete Fourier matrix ${\mathbf{V}}_N\in {\mathbb{C}}^{N\times N}$ formulated as ${\mathbf{L}}_N={\mathbf{V}}_N\mathrm{\Lambda }{\mathbf{V}}_N^T$ since it is circulant. Therefore, the Laplacian matrix

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \mathbf{L}={\mathbf{L}}_N\otimes {\mathbf{I}}_N+{\mathbf{I}}_N\otimes {\mathbf{L}}_N\end{array}\end{array}$$

(22)

is circulant as well, and it can be diagonalized by the discrete Fourier matrix $\mathbf{V}={\mathbf{V}}_N^{\otimes 2}$ [39,40]. The discrete Fourier transform (DFT) of the vectorized 2-dimensional signal $\mathrm{vec}\left(\mathit{x}\right)$ is implemented by $\mathrm{vec}\left(\widehat{\mathit{X}}\right)={\mathbf{V}}^T\mathrm{vec}\left(\mathit{X}\right)$.

Specifically, for a 2nd-order HGS ${\mathit{s}}_{\left(2\right)}$, we can treat it as a special case of general 2-dimensional signals. If $\mathrm{mat}\left({\mathbf{G}}_{\left(4\right)}\right)$ is a diagonal matrix, only all diagonal entries need to be considered as spectral coefficients. The total variation and the signal transformation we proposed are consistent with those of the above vectorized 2-dimensional signal $\mathrm{vec}\left(\mathit{X}\right)$. The necessary and sufficient condition for $\mathrm{mat}\left({\mathbf{G}}_{\left(4\right)}\right)$ to be diagonal is that ${\mathbf{G}}_{\left(4\right)}$ is superdiagonal since ${\mathbf{G}}_{\left(4\right)}$ is supersymmetric. The proof can be found in Appendix B. In other words, we assume that the hypergraph Laplacian ${\mathbf{L}}_{\left(4\right)}$ can be orthogonally superdiagonalized by the basis $\mathbf{U}$ among all 4 modes in this special case.

4. Hypergraph Filters

In general, a hypergraph filter is a system which takes an HGS $\mathit{s}$ as the input and obtains another HGS $\tilde{\mathit{s}}$ as the output. The novel HGFT proposed in Section 3.2 provides both vertex and spectral perspectives for HGS filtering. We now consider a hypergraph filter $\mathbf{F}\in {\mathbb{R}}^{{\left|\mathcal{V}\right|}^M\times {\left|\mathcal{V}\right|}^M}$ and denote its spectral form by $\widehat{\mathbf{F}}\in {\mathbb{R}}^{{\left|\mathcal{V}\right|}^M\times {\left|\mathcal{V}\right|}^M}$. The filter in the two domains can be viewed as a function of ${\mathbf{L}}_{\left(c\right)}$ and ${\mathbf{G}}_{\left(c\right)}$, respectively. In the vertex domain, we can filter a high-order HGS directly by

$$\begin{array}{c}\hfill \begin{array}{ccccc}\hfill \left(\mathbf{vertex}\mathbf{domain}\right)& & \hfill {\tilde{\mathit{s}}}_{\left(M\right)}& =\mathbf{F}{\mathit{s}}_{\left(M\right)},\hfill & \hfill \end{array}\end{array}$$

(23)

while we filter a spectral-domain high-order signal by

$$\begin{array}{c}\hfill \begin{array}{ccccc}\hfill \left(\mathbf{spectral}\mathbf{domain}\right)& & \hfill {\widetilde{\widehat{\mathit{s}}}}_{\left(M\right)}& =\widehat{\mathbf{F}}{\widehat{\mathit{s}}}_{\left(M\right)}.\hfill & \hfill \end{array}\end{array}$$

(24)

By substituting the HGFT and IHGFT into (24), any spectral filtering can be implemented in the vertex domain as

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \mathbf{F}={\mathbf{U}}^{\otimes M}\widehat{\mathbf{F}}{\left({\mathbf{U}}^{\otimes M}\right)}^T.\end{array}\end{array}$$

(25)

For $M=1$, the hypergraph $\mathcal{H}$ degenerates into a graph, and (23) and (24) represent graph filtering in the vertex and spectral domains, respectively.

Filtering in either of the two domains takes linear transformations of high-order components as the results. However, the subset ${\mathcal{S}}_{\left(M\right)}$ is defined to contain all vector forms of the Mth-order $\left|\mathcal{V}\right|$-dimension supersymmetric rank-one tensors. Obviously, it is not a subspace and may not contain the column space of $\mathbf{F}$. In this case, the subset ${\mathcal{S}}_{\left(M\right)}$ does not satisfy the rules for vector addition and multiplication by real scalars. Thus, after implementing the linear combination of high-order components, the filtering result ${\tilde{\mathit{s}}}_{\left(M\right)}$ may not lie in the subset ${\mathcal{S}}_{\left(M\right)}$ anymore. It may take an invalid form and not be able to reduce to the original 1st-order form as the filter output. In this case, we need to add the rank-one approximation (ROA) of $\mathrm{ten}\left({\tilde{\mathit{s}}}_{\left(M\right)}\right)$ into the filtering process to obtain a valid high-order signal form. The ROA of the supersymmetric tensor $\mathrm{ten}\left({\tilde{\mathit{s}}}_{\left(M\right)}\right)$ in the least-squares sense [41] solves a minimization problem formulated as

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \begin{array}{cc}\hfill (\widehat{\lambda },\widehat{\mathit{u}})=& \underset{\lambda \in \mathbb{R},\mathit{u}\in {\mathbb{R}}^{\left|\mathcal{V}\right|}}{argmin}{\parallel \mathrm{ten}\left({\tilde{\mathit{s}}}_{\left(M\right)}\right)-\lambda {\mathit{u}}^{⊠M}\parallel }_2^2\hfill \\ \hfill & \begin{array}{ccc}\hfill \mathrm{s}.\mathrm{t}.& \hfill {\mathit{u}}^T\mathit{u}& \hfill =1\end{array}\hfill \end{array}\end{array}\end{array}$$

(26)

and arrives at the solution

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \mathrm{ROA}\left({\tilde{\mathit{s}}}_{\left(M\right)}\right)=\widehat{\lambda }{\widehat{\mathit{u}}}^{\otimes M},\end{array}\end{array}$$

(27)

which preserves the most dominated supersymmetric rank-one component of $\mathrm{ten}\left({\tilde{\mathit{s}}}_{\left(M\right)}\right)$ in the least-squares sense. The relationship between the valid forms of (23) and (24) still maintains the correspondence of signals in both domains formulated as

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \mathrm{ROA}\left({\tilde{\mathit{s}}}_{\left(M\right)}\right)={\mathbf{U}}^{\otimes M}\mathrm{ROA}\left({\widetilde{\widehat{\mathit{s}}}}_{\left(M\right)}\right)\end{array}\end{array}$$

(28)

where $\mathrm{ROA}(·)$ specifically denotes the equivalent vector operation of the tensor ROA, and the proof is given in Appendix C. In this way, we can ensure that taking the original HGS $\mathit{s}$ as an input, the hypergraph filter can obtain a vector of the same size as its output. Outputs of a vertex-domain filter $\mathbf{F}$ and a spectral filter $\widehat{\mathbf{F}}$ can be respectively obtained by

$$\begin{array}{c}\hfill \begin{array}{ccccc}\hfill \left(\mathbf{vertex}\mathbf{domain}\right)& & \hfill \tilde{\mathit{s}}& ={\left(\mathrm{ROA}\left({\tilde{\mathit{s}}}_{\left(M\right)}\right)\right)}^{\otimes 1/M}\hfill & \hfill \end{array}\end{array}$$

(29)

and

$$\begin{array}{c}\hfill \begin{array}{ccccc}\hfill \left(\mathbf{spectral}\mathbf{domain}\right)& & \hfill \widetilde{\widehat{\mathit{s}}}& ={\left(\mathrm{ROA}\left({\widetilde{\widehat{\mathit{s}}}}_{\left(M\right)}\right)\right)}^{\otimes 1/M},\hfill & \hfill \end{array}\end{array}$$

(30)

where ${(·)}^{\otimes 1/M}$ denotes the inverse operation of the Kronecker products of M identical vectors. If M is odd, we can obtain the value of the ith entry by plugging $i_1=\cdots =i_M=i$ into (10) and calculating its $1/M$th power. However, if M is even, we can only obtain the absolute value of the ith entry because of the sign ambiguity. The relative sign between signals at vertices $v_i$ and $v_j$ can be obtained by entries in (10) with ${\left\{i_m\right\}}_{m=1}^M$ composed of odd numbers of i and j. The final signs of signals can be determined according to signs of observations.

Example 7.

For the operator ${(·)}^{\otimes 1/M}$ with an even M, according to (10), it can be learned from ${\left(\mathrm{ROA}\left({\tilde{\mathit{s}}}_{\left(M\right)}\right)\right)}_1={\tilde{s}}_1^M=1$ and ${\left(\mathrm{ROA}\left({\tilde{\mathit{s}}}_{\left(M\right)}\right)\right)}_{1+{\sum }_{m=1}^M{\left|\mathcal{V}\right|}^{M-m}}={\tilde{s}}_2^M=1$ that $|{\tilde{s}}_1|=|{\tilde{s}}_2|=1^{1/M}=1$. The signs of ${\tilde{s}}_1$ and ${\tilde{s}}_2$ are the same if ${\left(\mathrm{ROA}\left({\tilde{\mathit{s}}}_{\left(M\right)}\right)\right)}_{1+{\sum }_{m=2}^M{\left|\mathcal{V}\right|}^{M-m}}={\tilde{s}}_1{\tilde{s}}_2^{M-1}=1$. The signs of ${\tilde{s}}_1$ and ${\tilde{s}}_2$ are different if ${\left(\mathrm{ROA}\left({\tilde{\mathit{s}}}_{\left(M\right)}\right)\right)}_{1+{\sum }_{m=2}^M{\left|\mathcal{V}\right|}^{M-m}}={\tilde{s}}_1{\tilde{s}}_2^{M-1}=-1$. We can judge the final sign of the HGS $\tilde{\mathit{s}}$ according to signs of observations.

We now discuss two specific forms of hypergraph filters. One is polynomial filters which are based on the hypergraph Laplacians and can be operated in the vertex domain directly. The other one is reducible filters which can ensure that all high-order signals in the whole processing task are in valid forms and are able to degenerate into their 1st-order forms.

4.1. Polynomial Filters Based on the Hypergraph Laplacians

Polynomial hypergraph filters are a kind of common and handy filter which can be directly implemented in the vertex domain of the HGS. Taking vertex-domain signals as the input, we can construct a hypergraph polynomial filter based on the hypergraph Laplacians in Figure 4 formulated by

$$\begin{array}{c}\hfill \begin{array}{ccccc}\hfill \left(\mathbf{vertex}\mathbf{domain}\right)& & \hfill \tilde{\mathit{s}}& ={\left(\mathrm{ROA}\left(\sum _{k=0}^K{a}_k\mathrm{mat}{\left({\mathbf{L}}_{\left(c\right)}\right)}^k{\mathit{s}}_{\left(M\right)}\right)\right)}^{\otimes 1/M}={\left(\mathrm{ROA}\left(\sum _{k=0}^K{a}_k{\tilde{\mathit{s}}}_{\left(M\right)}^{\left(k\right)}\right)\right)}^{\otimes 1/M},\hfill & \hfill \end{array}\end{array}$$

(31)

where K is the order of the polynomial filter, ${\left\{a_k\right\}}_{k=0}^K$ are the filter coefficients and ${\tilde{\mathit{s}}}_{\left(M\right)}^{\left(k\right)}$ is the kth-order term of the hypergraph Laplacian filtering result. The whole filtering process can be divided into two parts. To obtain the kth-order term ${\tilde{\mathit{s}}}_{\left(M\right)}^{\left(k\right)}$, we process the high-order signal ${\mathit{s}}_{\left(M\right)}$ by ${\mathbf{L}}_{\left(c\right)}$ for k times. Then, we take the linear combination of results of each order according to the filter coefficients, perform a rank-one approximation to obtain a valid high-order result and finally reduce it into its 1st-order form as the output of the filter. Regardless of whether the filtering result is valid or not, the computational cost of simply filtering an Mth-order $\left|\mathcal{V}\right|$-vertex hypergraph signal is $O\left(\frac{(K+1)K}{2}\right|\mathcal{V}{|}^{2M})$, namely, $O\left(\right|\mathcal{V}{|}^c)$. The computational cost of the algorithm of the ROA of an Mth-order $\left|\mathcal{V}\right|$-vertex hypergraph signal is $O\left(N_{\mathrm{iter}}{\sum }_{i=1}^M\right|\mathcal{V}{|}^i)$, namely, $O\left(\right|\mathcal{V}{|}^M)$, where $N_{\mathrm{iter}}$ denotes the number of iterations in the algorithm of the tensor ROA. Therefore, the computational cost of the high-order polynomial filter is $O\left(\right|\mathcal{V}{|}^c)$.

From the spectral perspective, by (28), we accordingly have

$$\begin{array}{c}\hfill \begin{array}{ccccc}\hfill \left(\mathbf{spectral}\mathbf{domain}\right)& & \hfill \widetilde{\widehat{\mathit{s}}}& ={\left(\mathrm{ROA}\left(\sum _{k=0}^K{a}_k\mathrm{mat}{\left({\mathbf{G}}_{\left(c\right)}\right)}^k{\widehat{\mathit{s}}}_{\left(M\right)}\right)\right)}^{\otimes 1/M}.\hfill & \hfill \end{array}\end{array}$$

(32)

As we have discussed before, the hypergraph Laplacian ${\mathbf{L}}_{\left(c\right)}$ can be treated as a high-order difference operator, and ${\mathbf{G}}_{\left(c\right)}$ takes high values on the positions corresponding to rapidly oscillating components in the tensor decomposition of ${\mathbf{L}}_{\left(c\right)}$. Therefore, ${\mathbf{L}}_{\left(c\right)}$ can be regarded as a high-pass filter focusing on relatively greatly varying components of signals. We can utilize ${\mathbf{L}}_{\left(c\right)}$ to process ${\mathit{s}}_{\left(M\right)}$ and mostly obtain the high-frequency components. Thus, the polynomial filter takes the ROA of the linear combination of the high-order HGS and its multi-order high-pass filtering results as the output.

When the filter coefficients ${\left\{a_k\right\}}_{k=0}^K$ take different values, we can obtain filters with distinct spectral properties for different applications. For instance, if smoothly varying components are more important and informative to the tasks, we can adjust the coefficients to remove the rapidly oscillating component from the input signal ${\mathit{s}}_{\left(M\right)}$, namely, ${\tilde{\mathit{s}}}_{\left(M\right)}^{\left(0\right)}$, by utilizing the multi-order high-pass filtering results ${\left\{{\tilde{\mathit{s}}}_{\left(M\right)}^{\left(k\right)}\right\}}_{k=1}^K$ in (31). The above operation can be considered as a low-pass filter.

4.2. Reducible Filters

Reducible filters provide a way to ensure the validity of the high-order HGS throughout the whole filtering process. Thus, reducible filters do not need the ROA of the high-order HGS and allow the whole signal processing task to be equivalently implemented in a 1st-order way. More precisely, the reducibility requires filters in either of the two domains to be the Kronecker products of M identical matrices. We then can represent the spectral-domain filter by $\widehat{\mathbf{F}}={\widehat{\mathbf{F}}}_m^{\otimes M}$, where ${\widehat{\mathbf{F}}}_m\in {\mathbb{R}}^{\left|\mathcal{V}\right|\times \left|\mathcal{V}\right|}$. By substituting the spectral form into (25), the corresponding vertex-domain form is $\mathbf{F}={\left(\mathbf{U}{\widehat{\mathbf{F}}}_m{\mathbf{U}}^T\right)}^{\otimes M}={\mathbf{F}}_m^{\otimes M}$. Under this condition, hypergraph filtering in the vertex domain (23) and the spectral domain (24) can be precisely written as

$$\begin{array}{c}\hfill \begin{array}{ccccc}\hfill \left(\mathbf{vertex}\mathbf{domain}\right)& & \hfill {\tilde{\mathit{s}}}_{\left(M\right)}& ={\mathbf{F}}_m^{\otimes M}{\mathit{s}}_{\left(M\right)}={\mathbf{F}}_m^{\otimes M}{\mathit{s}}^{\otimes M}={\left(\mathbf{U}{\widehat{\mathbf{F}}}_m{\mathbf{U}}^T\mathit{s}\right)}^{\otimes M}\hfill & \hfill \end{array}\end{array}$$

(33)

and

$$\begin{array}{c}\hfill \begin{array}{ccccc}\hfill \left(\mathbf{spectral}\mathbf{domain}\right)& & \hfill {\widetilde{\widehat{\mathit{s}}}}_{\left(M\right)}& ={\widehat{\mathbf{F}}}_m^{\otimes M}{\widehat{\mathit{s}}}_{\left(M\right)}={\left({\widehat{\mathbf{F}}}_m\mathit{s}\right)}^{\otimes M},\hfill & \hfill \end{array}\end{array}$$

(34)

and thus can be respectively reduced to

$$\begin{array}{c}\hfill \begin{array}{ccccc}\hfill \left(\mathbf{vertex}\mathbf{domain}\right)& & \hfill \tilde{\mathit{s}}& ={\mathbf{F}}_m\mathit{s}=\mathbf{U}{\widehat{\mathbf{F}}}_m{\mathbf{U}}^T\mathit{s}\hfill & \hfill \end{array}\end{array}$$

(35)

and

$$\begin{array}{c}\hfill \begin{array}{ccccc}\hfill \left(\mathbf{spectral}\mathbf{domain}\right)& & \hfill \widetilde{\widehat{\mathit{s}}}& ={\widehat{\mathbf{F}}}_m\widehat{\mathit{s}}\hfill & \hfill \end{array}\end{array}$$

(36)

as Figure 5 shows.

As is mentioned in Section 2, the hypergraph $\mathcal{H}$ is undirected and does not take into account edge-dependent vertex weights [42], which means that all the c vertices in a hyperedge are treated equivalently. We hope that in each hyperedge, the way of signal transmission among the c vertices in the filtering process is the same for any direction to any vertex. Therefore, we constrain the tensor form of the filter $\widehat{\mathbf{F}}$ to be supersymmetric. We then utilize the following proposition which is proven in Appendix D to filter the 1st-order HGS.

Proposition 2.

A real tensor $\mathrm{ten}\left(\widehat{\mathbf{F}}\right)={\widehat{\mathbf{F}}}_m^{⊠M}$ is supersymmetric for $M\ge 2$ if and only if ${\widehat{\mathbf{F}}}_m$ is a real symmetric rank-one matrix or a zero matrix.

We now consider that a degenerate spectral filter ${\widehat{\mathbf{F}}}_m$ to be designed is a real symmetric rank-one matrix and takes the form ${\widehat{\mathbf{F}}}_m={\lambda }_m\widehat{\mathit{f}}{\widehat{\mathit{f}}}^T$ where the unit vector $\widehat{\mathit{f}}\in {\mathbb{R}}^{\left|\mathcal{V}\right|}$ and the scalar ${\lambda }_m\in \mathbb{R}$. The degenerate form of the vertex-domain reducible filtering (35) can then be rewritten as

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \tilde{\mathit{s}}={\lambda }_m\mathbf{U}\widehat{\mathit{f}}{\widehat{\mathit{f}}}^T{\mathbf{U}}^T\mathit{s}={\lambda }_m\left(\mathbf{U}\widehat{\mathit{f}}\right){\left(\mathbf{U}\widehat{\mathit{f}}\right)}^T\mathit{s}.\end{array}\end{array}$$

(37)

It can be learned from (37) that ${\widehat{f}}_i$ (the ith entry of $\widehat{\mathit{f}}$) corresponds to the ith component of the basis. By designing all the coefficients in $\widehat{\mathit{f}}$, we obtain a unit vector $\mathbf{U}\widehat{\mathit{f}}$ which is a signal with multiple spectral components. The filtering process is to project the signal $\mathit{s}$ onto the unit vector $\mathbf{U}\widehat{\mathit{f}}$ and then to multiply the projecting result by a coefficient ${\lambda }_m$ which includes the sign information and the amplitude gain of the filter. The smoothness property of the signal $\mathbf{U}\widehat{\mathit{f}}$ directly determines the spectral components retained in the filtering process.

In general, as a function of ${\mathbf{G}}_{\left(c\right)}$, the original reducible filter $\widehat{\mathbf{F}}={\lambda }_m^M{\left(\widehat{\mathit{f}}{\widehat{\mathit{f}}}^T\right)}^{\otimes M}$ considers spectra corresponding to both valid and invalid high-order basis vectors and preserves a valid HGS form throughout the whole process simultaneously. Moreover, unlike some other filters, reducible filters can degenerate into a 1st-order form and thus provide outputs with intuitive spectral interpretations without the additional ROA.

While designing filters such as low-pass, high-pass, or band-pass filters, we can adjust the filter coefficients $\widehat{\mathit{f}}$ and ${\lambda }_m$ according to the practical demand. For example, the output of a low-pass filter tends to mainly concentrate on smooth HGS components represented by the first few basis vectors.

5. Application

As one of the widely studied supervised learning tasks, label learning studies a problem where each instance corresponds to a real-world agent associated with a label. The problem is to learn a function that predicts labels for unobserved instances according to some given observations. If features of all instances take the form of hypergraph topologies, the label learning task refers to hypergraph label learning specifically.

Hypergraph label learning has been widely applied to various scenarios such as automatic image annotation [43], visual classification of 2D or 3D objects [44,45,46], recommender systems [47,48], etc. Existing methods mostly utilize matrix-based or tensor-based forms to represent and process the HGS.

5.1. Hypergraph Label Learning Model

We model hypergraph labels by an HGS and implement hypergraph label learning in two steps. We first estimate the HGS by solving the following minimization problem

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \underset{\mathit{s}\in {\mathbb{R}}^{\left|\mathcal{V}\right|}}{min}\ell (\mathit{s},\mathit{y},\mathrm{\Psi })+\lambda \mathrm{\Omega }\left(\mathit{s}\right),\end{array}\end{array}$$

(38)

where the loss function ℓ measures the estimation error of known labels $\mathit{y}$, the regularization term $\mathrm{\Omega }$ aims to avoid overfitting, the nonnegative parameter $\lambda$ makes a trade-off between the above two terms, and $\mathrm{\Psi }$ is the sampling operator of the training set. We then set a threshold, divide all vertices into two categories according to the estimated signal and finally obtain the labels of the test set.

Concretely, we here provide a label learning method by directly filtering the 1st-order HGS, namely, using the degenerate form of reducible filters (37) by setting the filter coefficients ${\lambda }_m$ and $\widehat{\mathit{f}}$ appropriately. We assume that the HGS evolves smoothly over the topology and lies in a low-dimension vector space since we construct hypergraphs based on the vertex similarity and correlation. We then consider recovering signals by constraining it in a low-dimension linear subspace spanned by the first few vectors of the orthonormal basis. Therefore, given the number of vertices with known labels N, the last $\left|\mathcal{V}\right|-N$ entries of $\widehat{\mathit{f}}$ are constrained to be zero, and the 1st-order HGS to be recovered can be represented by

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \tilde{\mathit{s}}={\lambda }_m\mathbf{U}\widehat{\mathit{f}}{\widehat{\mathit{f}}}^T{\mathbf{U}}^T\mathit{s}=\mathbf{U}\mathrm{\Phi }\mathbf{\alpha },\end{array}\end{array}$$

(39)

where $\mathrm{\Phi }={[{\mathbf{I}}_N,{\mathbf{0}}_{N\times \left(\right|\mathcal{V}|-N)}]}^T\in {\mathbb{R}}^{\left|\mathcal{V}\right|\times N}$ is a binary matrix and $\mathbf{\alpha }={\lambda }_m{\mathrm{\Phi }}^T\widehat{\mathit{f}}{\widehat{\mathit{f}}}^T\widehat{\mathit{s}}\in {\mathbb{R}}^N$ is the first N entries of the output of the 1st-order spectral reducible filter. For the sake of simplicity, instead of designing the coefficients ${\lambda }_m$ and $\widehat{\mathit{f}}$ of the reducible filter, we can filter the HGS by learning $\mathbf{\alpha }$ whose dimension N is less than the number of the filter coefficients. However, a sufficiently large N possibly introduces the basis vector which is not smooth enough and tends to lead some large-variation components into the estimation of unknown labels. We thus consider to introduce a regularization term by constraining the values of $\mathbf{\alpha }$ and sacrificing a little estimation accuracy since an appropriate N takes different values under different specific situations. In our experiment, we choose the square loss of known labels as the loss function ℓ and the squared ${\ell }_2$-norm of $\mathit{\alpha }$ as the regularization term $\mathrm{\Omega }$. The optimization problem (38) can then be rewritten as

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \underset{\mathbf{\alpha }\in {\mathbb{R}}^N}{min}{\parallel \mathrm{\Psi }\mathbf{U}\mathrm{\Phi }\mathbf{\alpha }-\mathit{y}\parallel }_2^2+\lambda {\parallel \mathbf{\alpha }\parallel }_2^2.\end{array}\end{array}$$

(40)

5.2. Experimental Setups and Results

In this subsection, we provide two simulation examples over the acute inflammations dataset [49] and the iris dataset from the UCI Machine Learning Repository [50]. For the two datasets, we model them by 4-uniform weighted undirected hypergraphs based on the Euclidean distances of agents in their attribute vector space. The more similar the attribute vectors of any group $\mathcal{G}$ of 4 agents are, the more similar or correlated those agents seem to be and the more likely they are to be in one hyperedge. Therefore, in hypergraph topology constructions, we denote the arithmetic mean and the variance of the distance between any two agents in the group $\mathcal{G}$ by ${\mathrm{dist}}_{\mathrm{am}}=\frac{1}{\left(\genfrac{}{}{0pt}{}{4}{2}\right)}{\sum }_{\{v_i,v_j\}\in \mathcal{G}}\mathrm{dist}(v_i,v_j)$ and ${\mathrm{dist}}_{\mathrm{var}}=\frac{1}{\left(\genfrac{}{}{0pt}{}{4}{2}\right)}{\sum }_{\{v_i,v_j\}\in \mathcal{G}}{(\mathrm{dist}(v_i,v_j)-{\mathrm{dist}}_{\mathrm{am}})}^2$, respectively. We then consider the values of ${\mathrm{dist}}_{\mathrm{am}}$ and ${\mathrm{dist}}_{\mathrm{var}}$, and we accordingly set two thresholds ${\tau }_{\mathrm{am}}$ and ${\tau }_{\mathrm{var}}$ of the above two values to determine the existence of a hyperedge. We set ${\tau }_{\mathrm{am}}$ for two datasets equal to $0.5$ and $0.07$, respectively, and set ${\tau }_{\mathrm{var}}$ for both datasets to select the $50\%$ smallest-variance hyperedges of all the hyperedges which have already met the threshold ${\tau }_{\mathrm{am}}$. In addition, each vertex and its nearest group members must be in a hyperedge to ensure that no vertex is isolated. The weight of each hyperedge can be calculated by a Gaussian kernel function of ${\mathrm{dist}}_{\mathrm{am}}$ formulated as $w=\exp (-\frac{{\mathrm{dist}}_{\mathrm{am}}^2}{2{\sigma }^2})$ where parameters $\sigma$ of the two datasets are set to $2\sqrt{2}$ and $1/\sqrt{2}$ respectively. Here, we provide 15-vertex sub-hypergraphs as examples for both datasets in Figure 6. The signal defined at each vertex is the classification label represented by a scalar (for a two-class dataset) or a three-dimensional vector (for a three-class dataset) taking the values from $\{0,1\}$, and the label classification threshold is set to $0.5$.

We compare the proposed factorization-based method with other three factorization-based methods. Given the same hypergraph topology, HGSP’20 [25] obtains its HGFT basis by the orthogonal CP decomposition [26] of the adjacency tensor in the HGSP framework. As one of the graph approximation methods [33], the method in [32] projects high-order interactions onto pairwise ones and defines a normalized hypergraph Laplacian matrix (HGLM) according to the same hypergraph topology. We can obtain an orthonormal basis by the eigendecomposition of the Laplacian matrix and we denote the method by HGLM’06. Focusing on pairwise interactions from the beginning, the graph method directly constructs a graph according to the same threshold ${\tau }_{\mathrm{am}}$ and the same weight function, ensures that no vertex is isolated, and uses the GFT basis for the task.

We first consider the disease diagnosis over the acute inflammations dataset with $\left|\mathcal{V}\right|=120$ potential patients and two diseases (as two labels) for each potential patient. The manifestation of the acute inflammation of the urinary bladder (denoted by label $d_1$) and acute nephritisis (denoted by label $d_2$) is provided as 6 attributes of each person such as temperature, the occurrence of nausea, etc. In each trial, we randomly choose 10–25 persons as agents with known labels and left the labels of the remaining persons unknown. We tested the learning accuracy for the two diseases (labels $d_1$ and $d_2$), respectively, by setting $\lambda =0$ and taking the average of 1000-trial results, as shown in Figure 7.

In Figure 7, the proposed method performs best in the diagnosis accuracy of unknown-label potential patients, which indicates that the proposed orthonormal basis owns the most spectral information of both two signals in this situation. The spectral forms of the two signals are relatively sparse, and the signal components of the two signals tend to concentrate more on the low-frequency basis vectors. The performance of HGSP’20 is the least satisfactory here, likely because of the uncertainty caused by the orthogonal CP decomposition on some occasions. HGLM’06 does not perform as well as the proposed method probably due to the projection of high-order interactions onto pairwise ones. The projections and the subsequent mathematical representations bring irreversible information loss to the analysis of the HGS. The graph method also performs very well in the learning of the two labels. The reason why the graph method outperforms HGLM’06 is possibly that HGLM’06 focuses on the hypergraph topology at first and attaches a much larger weight to the edge onto which is projected by multiple large-weight hyperedges by the direct summation of the hyperedge weights. That might not be suitable in this case. In general, the two methods describe and utilize different pairwise interactions from different starting points.

In addition to the binary classification for each label, we also provide a three-class simulation example over the iris dataset with $\left|\mathcal{V}\right|=150$ agents. We model the three-class label of each agent by a three-dimensional binary vector, as mentioned before, and we solve the optimization problem (40) for each dimension by setting $\lambda$ equal to 0, $0.007$ and $0.007$, respectively. We test the label learning accuracy of unknown-label agents with the number of known labels N ranging from 1 to 40 by averaging 1000-repetition results. The comparative results are shown in Figure 8.

It can be observed in Figure 8 that the proposed method generally achieves a higher accuracy than other methods. For a very few N, HGLM’06 performs well and the accuracy increases rapidly, which indicates that its first few basis vectors do greatly help with the classification task. Both the proposed method and HGLM’06 behave well for a relatively small N, which shows the advantage of focusing on high-order interactions. Unlike those two methods, the graph method does not perform so well and it seems to need more labels to achieve a better result when N takes relatively small values. For a larger N, the graph method obtains more labels and performs a lot better. In summary, the above results show the advantage of the proposed method in this scenario and demonstrate the advantage of mining high-order interactions, especially under the lack of known labels.

6. Conclusions

To capture high-order interactions in complex structured signal processing, we propose an HGS analysis framework based on the hyperedge-wise HOTV proposed in this paper. The HOTV can be regarded as a smoothness measure of the HGS, and the hypergraph Laplacian obtained from the HOTV can be regarded as a high-order difference operator of the HGS. According to the smoothness measure, we construct an orthonormal basis reflecting spectral information of signals over the topology. We further propose a new signal transformation (a novel HGFT) and introduce both the vertex and the spectral domains of the HGS. We then implement hypergraph filtering in both domains equivalently and provide two specific filter forms. Finally, we validate the advantages of the proposed framework in some scenarios using applications in label learning and some experimental results.