Leveraging Attribute Interaction and Self-Training for Graph Alignment via Optimal Transport

Abstract

Unsupervised alignment of two attributed graphs finds the node correspondence between them without any known anchor links. The recently proposed optimal transport (OT)-based approaches tackle this problem via Gromov–Wasserstein distance and joint learning of graph structures and node attributes, which achieve better accuracy and stability compared to previous embedding-based methods. However, it remains largely unexplored under the OT framework to fully utilize both structure and attribute information. We propose an Optimal Transport-based Graph Alignment method with Attribute Interaction and Self-Training (PORTRAIT), with the following two contributions. First, we enable the interaction of different dimensions of node attributes in the Gromov–Wasserstein learning process, while simultaneously integrating multi-layer graph structural information and node embeddings into the design of the intra-graph cost, which yields more expressive power with theoretical guarantee. Second, the self-training strategy is integrated into the OT-based learning process to significantly enhance node alignment accuracy with the help of confident predictions. Extensive experimental results validate the efficacy of the proposed model.

1. Introduction

Given two attributed graphs $G_s$ and $G_t$, the graph alignment problem finds the correspondence between the node sets of two graphs. It has numerous real-world applications, such as finding user identity matchings across social networks [1,2,3,4], aligning protein interaction networks [5,6,7], identifying common authors from different academic collaboration networks [8,9], and conducting entity alignment in various domains [10,11]. Existing studies of this problem can be categorized into supervised and unsupervised approaches, where the former [9,12,13] needs a set of matched node pairs (referred to as anchors) while the latter [14,15] conducts the alignment without any supervision. In cases where the anchors are hard to obtain, for example, subgraph counting [16] and structural alignment of molecules, we are motivated for unsupervised solutions only based on the similarities of graph structures and node attributes (i.e., node features).

One prominent category of unsupervised graph alignment approaches [13,17,18,19,20,21] adopts the idea of “Embed-Then-Cross-Compare” [14]. Specifically, the alignment is based on the explicitly learned node embeddings, and the goal is to minimize the distance of embeddings between the matched nodes. Representative methods (e.g., [17,22,23]) use graph neural networks (GNNs) to integrate both graph structures and node features to improve the expressiveness of node embeddings; for example, [17] employs adversarial training (GAN) [24] to minimize the distance between node pairs requiring alignment, while other approaches [22,23] achieve alignment through graph structure refinement. However, it is shown that directly aligning two embedding spaces without supervision is unstable and yields inferior performance [12].

Recently, a line of optimal transport (OT)-based methods [12,14,15] has been proposed for graph alignment, which demonstrates their advantages in prediction accuracy by formulating graph alignment as the OT problem. Generally speaking, given two probability distributions defined on the node sets, the OT algorithm (e.g., [25]) takes a non-negative cost function for node pairs as input and computes a probabilistic correspondence (referred to as the alignment matrix) so that the weighted total cost is minimized. Moreover, the alignment matrix must be a joint probabilistic distribution corresponding to the two marginal distributions for node sets. Therefore, the crux lies in the design of the cost function for node pairs, which is commonly implemented following the idea of Wasserstein distance (WD) [26] and Gromov–Wasserstein distance (GWD) [27]. It has been demonstrated that the specific choice of the cost function has a great impact on matching accuracy. For example, state-of-the-art performance is obtained by considering graph structures and node attributes jointly [14,15]. Nonetheless, we note that the idea of integrating structures and attributes into the OT framework is still in its infancy, leaving a large room for improving both model accuracy and efficiency.

In this paper, we focus on the unsupervised setting of graph alignment and propose an Optimal Transport-based Graph Alignment method with Attribute Interaction and Self-Training (PORTRAIT), which enhances the OT-based approaches from two perspectives. First of all, we observe that representative OT-based solutions (e.g., [14]) adopt attribute propagation (a.k.a. feature propagation) in the intra-graph cost design of the Gromov–Wasserstein learning process. More specifically, the feature embedding of each node is propagated to neighborhoods dimension by dimension, without interaction between different dimensions of node attributes. This leads to insufficient information exchange when two attribute dimensions have similar semantics. In contrast, our model employs a learnable linear transformation layer after feature propagation, which yields more expressive power with theoretical guarantee and still guarantees provable convergence. Meanwhile, to integrate richer graph structures and information, we aggregate the outputs of all convolutional layers into node embeddings, as each layer captures distinct hidden features from both local and global perspectives. Superior matching accuracy is achieved in the empirical study, coinciding with our theoretical analysis.

Second, the intra-graph cost represents the interactions between nodes within a single graph, and it plays a crucial role in determining matching accuracy [14]. By analyzing the variability of interaction strength among nodes in intra-graph cost, we can categorize nodes as either easier or harder to match. Using nodes with higher matching confidence as pseudo-labels, we propose a self-training strategy with our refined cost computation process that leverages structural-based node similarities to enhance existing OT-based methods. Our contributions are summarized as follows.

• We propose an unsupervised learning approach for graph alignment under the framework of Gromov–Wasserstein distance and optimal transport. Compared to existing OT-based methods, deeper graph structures and node information are incorporated in the design of the intra-graph cost, meanwhile, linear transformation of node attributes is integrated into cost design to facilitate attribute interaction across dimensions. It is proved to have more expressive power while sharing the convergence results. Moreover, we introduce a cost normalization layer to enhance the stability of model training.
• Inspired by the global interaction properties of nodes reflected in the intra-graph cost, by selecting high-confidence matched node pairs as pseudo-labels, we propose a self-training strategy that not only enhances our model but also improves the accuracy of other OT-based methods.
• Extensive experimental results demonstrate PORTRAIT’s outstanding performance, achieving a 5% improvement in Hits@1, while also exhibiting remarkable efficiency and robustness.

2. Preliminary

2.1. Problem Definition

We formally define the unsupervised graph alignment problem. Particularly, an undirected attributed graph G of n nodes is associated with the adjacency matrix $\mathbf{A}\in {\{0,1\}}^{n\times n}$ and node attribute matrix $\mathbf{X}\in {\mathbb{R}}^{n\times d}$. ${\mathbf{A}}_{ij}$ is set to 1 if there is an unweighted edge between the i-th and j-th nodes.

Definition 1

(Unsupervised Graph Alignment). Given a pair of graphs $G_s=(V_s,E_s)$ and $G_t=(V_t,E_t)$ with adjacency matrices ${\mathbf{A}}_p$ and node features ${\mathbf{X}}_p$ ($p=\{s,t\}$), we need to predict a set of aligned (i.e., matched) node pairs $\left\{(u_i,v_k)\right|u_i\in V_s,v_k\in V_t\}$ without any observed correspondence, where $u_i$ and $v_k$ are corresponding nodes across two graphs.

Note that for the supervised setting of the problem, we additionally have a set of known matchings between two graphs which are referred to as anchors, and need to predict the matchings for the remaining nodes. In this paper, we focus on the optimal transport-based solutions for graph alignment as they achieve state-of-the-art performance for both supervised (e.g., [12]) and unsupervised settings (e.g., [14,15]). We briefly introduce the optimal transport problem.

Definition 2

(Optimal Transport  (OT) Problem). We are given two distributions μ and ν on two finite sets $\mathcal{X}$ and $\mathcal{Y}$, and a cost function $C:\mathcal{X}\times \mathcal{Y}\to \mathbb{R}$. We have $\left|\mathcal{X}\right|=n_1$ and $\left|\mathcal{Y}\right|=n_2$. Kantorovich’s formulation of the optimal transport problem finds a solution ${\mathbf{T}}^{*}$ such that

$${\mathbf{T}}^{*}=\underset{\mathbf{T}\in \Pi (\mu ,\nu )}{inf}\sum _{x=1}^{n_1}\sum _{y=1}^{n_2}C(x,y)T(x,y)=\underset{\mathbf{T}\in \Pi (\mu ,\nu )}{inf}\langle \mathbf{C},\mathbf{T}\rangle .$$

(1)

where $\Pi (\mu ,\nu )$ denotes all possible joint distributions with the corresponding marginals equal to μ and ν. We use $\mathbf{C}\in {\mathbb{R}}^{n_1\times n_2}$ and $\mathbf{T}\in {\mathbb{R}}^{n_1\times n_2}$ to represent the cost and alignment matrix, respectively.

In practice, the OT problem is typically solved with entropic constraints for better efficiency [25]. Specifically, a regularization term is introduced (with $\lambda >0$):

$${\mathbf{T}}^{*}=\underset{\mathbf{T}\in \Pi (\mu ,\nu )}{inf}\langle \mathbf{C},\mathbf{T}\rangle -\lambda h\left(\mathbf{T}\right).$$

(2)

where $h\left(\mathbf{T}\right)=-{\sum }_{i,k}\mathbf{T}(i,k)\log \mathbf{T}(i,k)$ is the entropy of $\mathbf{T}$. The solution to Equation (2) has the form of

$${\mathbf{T}}^{*}=Diag\left(\mathbf{a}\right)\exp (-{\displaystyle \frac{\mathbf{C}}{\lambda }})Diag\left(\mathbf{b}\right).$$

(3)

In particular, $\mathbf{a}$ and $\mathbf{b}$ are adjustable vectors depending on each other, while ${\mathbf{T}}^{*}$ has a unique solution as the objective is convex [25]. Note that we have ${\sum }_{i,k}\mathbf{T}(i,k)=1$ because it is a joint probability distribution, while $\mathbf{T}(i,k)$ is mostly determined by $\mathbf{C}(i,k)$ and they are in reverse correlation.

Without loss of generality, let $n_1=\left|V_s\right|$ and $n_2=\left|V_t\right|$; we assume that $n_1\le n_2$. To predict the correspondence between $V_s$ and $V_t$ following almost all previous works, for each $u_i\in V_s$, we choose $M\left(u_i\right)={argmax}_{v_k}\mathbf{T}(i,k)$, where $M\left(u\right)$ (resp. $M\left(v\right)$) is the matched vertex of u (resp. v) in the other graph.

Table 1. Table of notations.

Notation	Description
$G_s,G_t$	The two attributed graphs to be aligned
${\mathbf{A}}_s,{\mathbf{A}}_t$	The adjacency matrices of $G_s$ and $G_t$
${\mathbf{X}}_s,{\mathbf{X}}_t$	The node attributes
$n_1,n_2$	The number of nodes in $G_s$ and $G_t$ ($n_1\le n_2$)
${\mathbf{C}}_s,{\mathbf{C}}_t$	The intra-graph cost matrices for $G_s$ and $G_t$
${\mathbf{C}}_{gwd},{\mathbf{C}}_{int}$	The inter-graph cost matrix (of size $n_1\times n_2$)
$\mathbf{T}$	The alignment matrix

lists the frequently used notations throughout the paper.

2.2. State-of-the-Art Methods

We discuss the representative methods for graph alignment under both unsupervised and supervised settings and include them as the main competitors of our approach.

2.2.1. Unsupervised Graph Alignment

Representative methods for the unsupervised setting can be roughly divided into two categories. One category [13,17,18,19,20,21] follows the idea of “Embed-Then-Cross-Compare”, i.e., they first generate embeddings for all nodes in both graphs and then seek a solution to minimize the distance between those embeddings of matched nodes.

• WAlign [17]: The algorithm first employs lightweight graph neural networks (GNNs), e.g., [28], to compute the embedding ${\mathbf{z}}_u$ of each node u, and then treats unsupervised graph alignment as minimizing the following objective:

  $$M^{*}=\underset{\left|M\right|=n_1}{argmin}\sum _{(u_i,v_k)\in M}\parallel {\mathbf{z}}_{u_i}-{\mathbf{z}}_{v_k}\parallel .$$

  (4)

The above equation can be interpreted as the Wasserstein distance (WD) [26]. Thus, inspired by Wasserstein GAN [24], WAlign proposes a GAN-based framework, where the discriminator minimizes $-\left({\sum }_{u_i\in V_s^{\prime }}{f}_w\left({\mathbf{z}}_{u_i}\right)-{\sum }_{v_k\in V_t^{\prime }}{f}_w\left({\mathbf{z}}_{v_k}\right)\right)$. The 1-Lip-schitz function $f_w$ is implemented by a neural network, based on which the pseudo corresponding node pairs are generated. The parameters of the GNN are updated by a similar loss function with an additional reconstruction term.

• GAlign [22]: The model incorporates the idea of data augmentation into the learning objective to obtain high-quality node embeddings. In particular, it encourages the node embeddings ${\mathbf{Z}}_p^{\left(k\right)}$ ($p=s,t$) of each GNN layer k to retain graph structures, as well as to be robust upon graph perturbation. Afterward, the alignment refinement procedure augments the original graph by first choosing the confident node matchings and then increasing the weight of their adjacency edges. This serves as an effective unsupervised heuristic to iteratively match node pairs.
• GTCAlign [23]: Arguably having the best accuracy among embedding-based solutions, it simplifies the idea of GAlign with the following observation. First, it uses a GNN with randomized parameters for node embedding computation, as it is intricate to design an embedding-based objective that directly corresponds to the goal of node alignment. Second, the iterative learning process is empowered by graph augmentation, i.e., to adjust edge weights according to confident predictions. By normalizing the node embeddings to length 1, the learning process gradually decreases the cosine angle between confident node matchings.

Another recent line of research [12,14,15,29,30] adopts optimal transport for graph alignment, which finds a probabilistic matching between two distributions on graphs and demonstrates better accuracy and robustness.

• GWL [15]: Xu et al. [15] present a graph matching framework by combining the graph-based optimal transport and node embedding learning. It predicts an alignment matrix $\mathbf{T}$ following Definition 2. Given the learnable node representations ${\mathbf{Z}}_s\in {\mathbb{R}}^{n_1\times d}$ and ${\mathbf{Z}}_t\in {\mathbb{R}}^{n_2\times d}$, GWL minimizes a combination of the Wasserstein discrepancy and the Gromov–Wasserstein discrepancy (GWD), while the latter shares similar ideas with graph matching. Specifically, the objective for the Wasserstein discrepancy is $L_{wd}=\langle \mathbf{K}({\mathbf{Z}}_s,{\mathbf{Z}}_t),\mathbf{T}\rangle$, where $\mathbf{K}(·,·)$ is a function to measure the distance based on two node embeddings. The loss corresponding to the Gromov–Wasserstein discrepancy is defined as

  $$L_{gwd}=\langle {\mathbf{C}}_{gwd},\mathbf{T}\rangle =\langle \mathbf{L}({\mathbf{C}}_s\left({\mathbf{Z}}_s\right),{\mathbf{C}}_t\left({\mathbf{Z}}_t\right),\mathbf{T}),\mathbf{T}\rangle ,$$

  (5)

  where the form of $\mathbf{L}\in {\mathbb{R}}^{n_1\times n_2}$ follows GWD:

  $$\begin{array}{c}\hfill \mathbf{L}({\mathbf{C}}_s,{\mathbf{C}}_t,\mathbf{T})(i,k)=\sum _{u_j\in G_s}\sum _{v_l\in G_t}{({\mathbf{C}}_s(i,j)-{\mathbf{C}}_t(k,l))}^2\mathbf{T}(j,l).\end{array}$$

  (6)

For GWL, the forms of intra-graph cost matrices ${\mathbf{C}}_s$ and ${\mathbf{C}}_t$ are hand-engineered, which is a combination of the adjacency information and the cosine distance based on node embeddings.

Intuitively, Equation (6) states that if $(u_i,v_k)$ and $(u_j,v_l)$ are two matched pairs, the distance between $u_i$ and $u_j$ in $G_s$ (i.e., ${\mathbf{C}}_s(i,j)$) is close to the distance between $v_k$ and $v_l$ in $G_t$ (i.e., ${\mathbf{C}}_t(k,l)$). Note that the matched pairs suggest a large value of $\mathbf{T}(i,k)$ (and $\mathbf{T}(j,l)$), indicating that $\mathbf{L}({\mathbf{C}}_s,{\mathbf{C}}_t,\mathbf{T})(i,k)$ is small. Consequently, the intra-graph costs need to be close, which establishes the correspondence between graph topologies. GWL leverages the proximal point method [31] to convert the GWD objective to an OT problem and alternatively learns the alignment matrix $\mathbf{T}$ and node embeddings ${\mathbf{Z}}_s$ and ${\mathbf{Z}}_t$. The convergence is theoretically guaranteed [15].

• SLOTAlign [14]: As the state-of-the-art unsupervised method, SLOTAlign improves existing GWD-based approaches (e.g., [15,32]) with a more careful design of the intra-graph costs ${\mathbf{C}}_s$ and ${\mathbf{C}}_t$. It introduces the module of multi-view structure modeling which incorporates graph structure, node features, and multi-hop information via an attribute propagation procedure. Then, it uses a few learnable parameters (with additional constraints) to compute the intra-graph cost from the weighted sum of the above information. Next, SLOTAlign adopts a similar procedure as GWL to learn the alignment matrix $\mathbf{T}$ by minimizing GWD (see Equation (5)).

2.2.2. Supervised Graph Alignment

Given a set of anchors as input, supervised graph alignment methods mostly focus on how to fully exploit the anchor information and propagate it to nearby nodes. Random Walk with Restart (RWR)-based solutions are widely adopted [9,13,33] while the very recent work [12] integrates the idea of label propagation with optimal transport.

• PARROT [12]: Given a set of anchors that $Q=\{(x_k,y_k),k\in [1,q]\}$ with $x_k\in G_s$ and $y_k\in G_t$, PARROT first generates the position-aware node embeddings for both graphs. For each anchor node $x_k$ (resp. $y_k$), it computes the RWR (a.k.a. Personalized PageRank, PPR [34]) vector on $G_s$ (resp. $G_t$):

  $${\mathbf{r}}_k^{\left(p\right)}=(1-\alpha ){\mathbf{W}}_p{\mathbf{r}}_k^{\left(p\right)}+\alpha {\mathbf{e}}_k^{\left(p\right)},p=s,t.$$

  (7)

Here, ${\mathbf{W}}_p={\mathbf{A}}_p^{⊺}{\mathbf{D}}_p^{-1}$ represents the transition matrix, with ${\mathbf{D}}_p$ being the diagonal degree matrix. The parameter $\alpha$ is referred to as the teleportation probability [34] and ${\mathbf{e}}_k^{\left(p\right)}$ is a one-hot vector with the $x_k$-th (resp. $y_k$-th) component to be 1. By concatenating all the vectors to get the $n_1\times q$-sized ${\mathbf{R}}_s=({\mathbf{r}}_1^{\left(s\right)},\dots ,{\mathbf{r}}_q^{\left(s\right)})$ and the $n_2\times q$-sized ${\mathbf{R}}_t=({\mathbf{r}}_1^{\left(t\right)},\dots ,{\mathbf{r}}_q^{\left(t\right)})$, each node in $G_s$ and $G_t$ obtains a q-dimensional positional embedding. Then, the inter-graph cost matrix ${\mathbf{C}}_{rwr}$ is computed by a two-step procedure:

$${\mathbf{C}}_{base}=\alpha e^{-{\mathbf{R}}_s{\mathbf{R}}_t^{⊺}}+(1-\alpha )e^{-{\mathbf{X}}_s{\mathbf{X}}_t^{⊺}}.$$

(8)

$$vec\left({\mathbf{C}}_{rwr}\right)=(1-\delta )\gamma {\mathbf{W}}_{s\otimes t}vec\left({\mathbf{C}}_{rwr}\right)+(1+\delta )vec\left({\mathbf{C}}_{base}\right).$$

(9)

To be more precise, it first computes a base case matrix ${\mathbf{C}}_{base}$ by comparing the positional embeddings as well as node attributes of two graphs, and then conducts an RWR-based propagation on the product graph $G_s\otimes G_t$ (i.e., conducting the Kronecker product on ${\mathbf{A}}_s$ and ${\mathbf{A}}_t$). Both $\delta$ and $\gamma$ are adjustable parameters, and $vec\left(\mathbf{C}\right)$ concatenates all the columns of $\mathbf{C}$ into an $n_1{n}_2$-dimensional vector.

With the inter-graph cost ${\mathbf{C}}_{rwr}$, PARROT transforms graph alignment to the OT problem, which is similar to GWL and SLOTAlign. The minimization of both WD and GWD is considered along with a few consistency regularization terms.

3. Our Approach

3.1. Model Overview

As demonstrated in Figure 1, our model is composed of three major phases. The attribute propagation and interaction phase takes graph structures (i.e., $\mathbf{A}$) and node attributes (i.e., $\mathbf{X}$) as input and computes the node embedding matrix $\mathbf{Z}$, which is then used to improve the cost computation in the following phase and to enhance model accuracy. The unsupervised Gromov–Wasserstein learning phase utilizes Gromov–Wasserstein distance [27] to address the issue of non-comparability arising from inconsistent graph topologies [14,15]. Both phases are combined to form an end-to-end pipeline that predicts the alignment matrix $\mathbf{T}$. Next, the self-training phase is applied to compute a revised prediction ${\mathbf{T}}^{\prime }$ to improve the matching accuracy and it can be applied to other existing OT-based methods (e.g., [14,15]). The pseudo-code is demonstrated in Algorithm 1, with the details of three phases described in the following subsections. Note that for each node $u_i$ in the smaller graph $G_s$, the algorithm returns $u_i$ with its matched node $M\left(u_i\right)$, where $M\left(u_i\right)$ is the node in $G_t$ with the largest matching probability ${\mathbf{T}}^{\prime }(i,k)$.

Algorithm 1: PORTRAIT.

Input: Two attributed graphs $G_s$ and $G_t$ with ${\mathbf{A}}_s$, ${\mathbf{X}}_s$, ${\mathbf{A}}_t$, and ${\mathbf{X}}_t$ Output: The set of predicted node matchings // Algorithm 2 ${\mathbf{Z}}_p\leftarrow \mathrm{AttrProp} \& \mathrm{Interact}({\mathbf{A}}_p,{\mathbf{X}}_p),p=\{s,t\}$; // Algorithm 3 $\mathbf{T}\leftarrow \mathrm{Unsuper}-\mathrm{GW}({\mathbf{A}}_s,{\mathbf{X}}_s,{\mathbf{Z}}_s,{\mathbf{A}}_t,{\mathbf{X}}_t,{\mathbf{Z}}_t)$; // Algorithm 4 ${\mathbf{T}}^{\prime }\leftarrow \mathrm{Self}-\mathrm{Training}(\mathbf{T},{\mathbf{A}}_s,{\mathbf{X}}_s,{\mathbf{A}}_t,{\mathbf{X}}_t)$; return$\{(u_i,M\left(u_i\right))\mid M\left(u_i\right)={argmax}_{v_k}{\mathbf{T}}^{\prime }(i,k),\forall u_i\in V_s\}$;

3.2. Attribute Propagation and Interaction

Compared to representative OT-based solutions [14,15], we add a preliminary step named attribute propagation and interaction in Figure 2 to enhance the cost computation of the next GW learning phase. As pointed out by [14], the alignment quality via OT heavily depends on the specific design of two intra-graph cost matrices. Given a graph G, let $\mathbf{X}\left(v\right)\in {\mathbb{R}}^d$ denote node v’s attribute. State-of-the-art solution [14] adopts the subgraph-view structure modeling process, which can be formulated as attribute propagation:

$${\mathbf{Z}}^{\prime }\left(v\right)=f\left(\mathbf{A}\right)\mathbf{X}\left(v\right),$$

(10)

where $\mathbf{A}$ is the adjacency matrix of G, and $f\left(\mathbf{A}\right)$ denotes the general transformation of $\mathbf{A}$. Specifically, it uses the following equation for propagation with $k\in [1,K]$ [14]:

$$f\left(\mathbf{A}\right)={\left({\tilde{\mathbf{D}}}^{-{\displaystyle \frac{1}{2}}}\tilde{\mathbf{A}}{\tilde{\mathbf{D}}}^{-{\displaystyle \frac{1}{2}}}\right)}^k,$$

(11)

where $\tilde{\mathbf{A}}=\mathbf{A}+\mathbf{I}$ and $\tilde{\mathbf{D}}$ is the diagonal matrix representing the degree of $\tilde{\mathbf{A}}$ (i.e., $\tilde{\mathbf{D}}=\mathrm{diag}\left(\tilde{\mathbf{A}}\mathbf{1}\right)$). To this end, for each dimension i of node v’s representation ${\mathbf{Z}}^{\prime }\left(v\right)$, ${\mathbf{Z}}^{\prime }(v,i)$ is augmented with the dimension-wise information of its k-hop neighbors.

However, two dimensions of the node attribute might have similar semantics, for example, when attributes are encoded in one-hot manner. In this case, it is reasonable to unify both dimensions in the latent space. Attribute propagation is incapable of encouraging information exchange across different dimensions of the node attributes. We propose an additional attribute interaction step with a learnable transformation matrix $\Theta \in {\mathbb{R}}^{(K+1)d\times d}$:

$$\mathbf{Z}\left(v\right)={\mathbf{Z}}^{\prime }\left(v\right)\Theta .$$

(12)

As $\mathbf{Z}(v,i)$ represents the linear combination of all the feature dimensions of ${\mathbf{Z}}^{\prime }\left(v\right)$, we implement the information exchange not only between different attributes of v but also between those of v and its k-hop neighbors. Although the solution is simple, it theoretically improves the expressive power of the Gromow–Wasserstein learning phase (see Section 3.3) and results in better matching accuracy in practice. The pseudo-code is shown in Algorithm 2.   

Algorithm 2: AttrProp&Interact.

Input: Adjacency matrix $\mathbf{A}$ and node attributes $\mathbf{X}$ Output: Node representation $\mathbf{Z}$ // Attribute Propagation ${\mathbf{Z}}^{\prime }\left(v\right)\leftarrow f\left(\mathbf{A}\right)\mathbf{X}\left(v\right)$; // Attribute Interaction $\mathbf{Z}\left(v\right)\leftarrow {\mathbf{Z}}^{\prime }\left(v\right)\Theta$; return $\mathbf{Z}\left(v\right)$;

3.3. Unsupervised Gromov–Wasserstein Learning

3.3.1. Cost Computation

We first describe the cost computation steps of the Gromov–Wasserstein (GW) learning framework (see Figure 3). Following existing approaches [14,15], we have intra-graph cost computation followed by inter-graph cost computation.

• Intra-graph cost computation. We adopt the multi-view modeling of intra-graph cost, which considers graph structure $\mathbf{A}$, node attributes $\mathbf{X}$, and a joint modeling of them via node representation $\mathbf{Z}$:

  $${\mathbf{C}}_p={\beta }_1^{\left(p\right)}·{\mathbf{A}}_p+{\beta }_2^{\left(p\right)}·{\mathbf{X}}_p{\mathbf{X}}_p^{⊺}+{\beta }_3^{\left(p\right)}·{\mathbf{Z}}_p{\mathbf{Z}}_p^{⊺},p=s,t$$

  (13)

  where ${\mathbf{C}}_s\in {\mathbb{R}}^{n_1\times n_1}$ and ${\mathbf{C}}_t\in {\mathbb{R}}^{n_2\times n_2}$ denote the intra-graph cost matrix of $G_s$ and $G_t$, respectively. Here, we use ${\mathit{\beta }}^{\left(p\right)}={({\beta }_1^{\left(p\right)},{\beta }_2^{\left(p\right)},{\beta }_3^{\left(p\right)})}^{⊺}$ to represent the learnable coefficients. Compared to [14], we compute $\mathbf{Z}$ through both attribute propagation and interaction (see Section 3.2), while simultaneously aggregating node representations from different convolutional layers to capture both local and global information, which enhances the model’s expressiveness (as demonstrated in Theorem 1). Specifically, it has the following concrete form:

  $$\begin{array}{cc}\hfill f({\mathbf{A}}_p,{\mathbf{X}}_p)& =\mathrm{Concat}({\mathbf{X}}_p,{\mathbf{W}}_p{\mathbf{X}}_p,\dots ,{\mathbf{W}}_p^K{\mathbf{X}}_p),\hfill \\ \hfill {\mathbf{Z}}_p& =\mathrm{Dropout}\left(\sigma \left(f({\mathbf{A}}_p,{\mathbf{X}}_p)\right)\right)\Theta .\hfill \end{array}$$

  (14)

We use $\mathbf{W}$ to represent the normalized adjacency matrix, i.e., $\mathbf{W}={\tilde{\mathbf{D}}}^{-{\displaystyle \frac{1}{2}}}\tilde{\mathbf{A}}{\tilde{\mathbf{D}}}^{-{\displaystyle \frac{1}{2}}}$, and $\sigma (·)$ denotes the activation function. Following our discussion in Section 3.2, $\Theta \in {\mathbb{R}}^{(K+1)d\times d}$ is the learnable transformation matrix in the attribute interaction step.

• Inter-graph cost computation. Given intra-graph cost matrices ${\mathbf{C}}_s$ and ${\mathbf{C}}_t$, this step follows the definition of Gromow–Wasserstein discrepancy (Equation (6)). We denote by ${\mathbf{C}}_{gwd}$ the $n_1\times n_2$-sized inter-graph cost matrix.

The following theorem states the discriminative power of our cost design backed by attribute propagation and interaction.

Theorem 1.

For the computation of the Gromov–Wasserstein discrepancy, with attribute interaction, the constructed intra-graph cost matrix has more discriminative power than when only attribute propagation is applied.

Proof. 

Let $M^{*}$ denote the ground truth node matching and assume that $(u_i,v_k),(u_j,v_k)\in M^{*}$ and $(u_{j^{\prime }},v_k)\notin M^{*}$. By the interpretation of GW discrepancy, ${\mathbf{C}}_s(i,j)$ and ${\mathbf{C}}_t(k,l)$ should be similar [14], say, $|{\mathbf{C}}_s(i,j)-{\mathbf{C}}_t(k,l)|<{\epsilon }_1$. Meanwhile, we should also have $|{\mathbf{C}}_s(i,j^{\prime })-{\mathbf{C}}_t(k,l)|\ge {\epsilon }_2$ with ${\epsilon }_2>{\epsilon }_1$.

Now, consider the case $dist(u_i,u_j)>K,dist(u_i,u_{j^{\prime }})>K$, and ${\mathbf{X}}_s{\left(u_i\right)}^{⊺}{\mathbf{X}}_s\left(u_j\right)={\mathbf{X}}_s{\left(u_i\right)}^{⊺}{\mathbf{X}}_s\left(u_{j^{\prime }}\right)=0$, for example; they are all one-hot vectors taking the values of 1 at different dimensions. Here, $dist(·,·)$ is the shortest distance between two nodes. Then, the first two terms of ${\mathbf{C}}_s(i,j)$ and ${\mathbf{C}}_s(i,j^{\prime })$ are 0 (see Equation (13)). Moreover, as $u_i$ and $u_j$ (resp. $u_{j^{\prime }}$) are with more than k-hops, after propagation of the one-hot attributes, we still have the third term ${\mathbf{Z}}_s^{\prime }{\left(u_i\right)}^{⊺}{\mathbf{Z}}_s^{\prime }\left(u_j\right)={\mathbf{Z}}_s^{\prime }{\left(u_i\right)}^{⊺}{\mathbf{Z}}_s^{\prime }\left(u_{j^{\prime }}\right)=0$, where ${\mathbf{Z}}_s^{\prime }=f\left({\mathbf{A}}_s\right){\mathbf{X}}_s$. Consequently, we have $|{\mathbf{C}}_s(i,j)-{\mathbf{C}}_t(k,l)|=|{\mathbf{C}}_s(i,j^{\prime })-{\mathbf{C}}_t(k,l)|=|{\mathbf{C}}_t(k,l)|$; that is, the cost design cannot discriminate the matched pair $(u_j,v_k)$ and unmatched node pair $(u_{j^{\prime }},v_k)$.

With attribute interaction, it is possible to learn the transformation matrix $\Theta$ so that ${\mathbf{Z}}_s{\left(u_i\right)}^{⊺}{\mathbf{Z}}_s\left(u_j\right)={\mathbf{Z}}_s^{\prime }{\left(u_i\right)}^{⊺}\Theta {\Theta }^{⊺}{\mathbf{Z}}_s^{\prime }\left(u_j\right)\ne 0$ even if ${\mathbf{Z}}_s^{\prime }{\left(u_i\right)}^{⊺}{\mathbf{Z}}_s^{\prime }\left(u_j\right)=0$, which also holds for $u_{j^{\prime }}$. For the same reason, it is possible to have $|{\mathbf{C}}_s(i,j)-{\mathbf{C}}_t(k,l)|\ne |{\mathbf{C}}_s(i,j^{\prime })-{\mathbf{C}}_t(k,l)|$, and the theorem follows.    □

Our cost computation differs from representative GW learning approaches [14,15] in the following aspects. We provide a more flexible multi-view modeling which enables a full interaction of graph structures and node attributes. Meanwhile, we take a step further to include attribute interaction, enhancing the model’s expressive power.

• Cost normalization. It has been observed that the Sinkhorn algorithm [25] invoked by the GW learning process might have numerical instability issues [35]. We observe that most terms ${({\mathbf{C}}_s(i,j)-{\mathbf{C}}_t(k,l))}^2$ are far less than 1 in practice. As the average value of $\mathbf{T}(j,l)$ is ${\displaystyle \frac{1}{n_1{n}_2}}$, the inter-graph cost for most node pairs are of very small values. We find that this phenomenon deteriorates the stability of GW learning. Hence, we employ a cost normalization module to rescale the inter-graph cost. Let $C_{max}={max}_{i,k}{\mathbf{C}}_{gwd}(i,k)$ and $C_{min}={min}_{i,k}{\mathbf{C}}_{gwd}(i,k)$, we have

  $${\mathbf{C}}_{gwd}(i,k)\leftarrow {\displaystyle \frac{{\mathbf{C}}_{gwd}(i,k)-C_{min}}{C_{max}-C_{min}}},\forall i,k.$$

  (15)

Note that after normalization, all the elements of ${\mathbf{C}}_{gwd}$ fall into $[0,1]$.

3.3.2. Learning Algorithm

We employ the widely used proximal point method [31,36] in existing OT-based solutions [12,14,15] to learn the parameters ${\Theta }^{\prime }=\{{\mathit{\beta }}^{\left(s\right)},{\mathit{\beta }}^{\left(t\right)},\Theta \}$ and the alignment matrix $\mathbf{T}$. It optimizes ${\Theta }^{\prime }$ and $\mathbf{T}$ alternatively with an iterative procedure. In particular, the objective is

$$\begin{array}{cc}\hfill & \underset{{\Theta }^{\prime },\mathbf{T}}{min}\langle {\mathbf{C}}_{gwd},\mathbf{T}\rangle =\underset{\mathbf{T}\in \prod (\mu ,\nu )}{min}\sum _{i,j,k,l}{\left|{\mathbf{C}}_s(i,j)-{\mathbf{C}}_t(k,l)\right|}^2\mathbf{T}(i,k)\mathbf{T}(j,l).\hfill \\ \hfill & \mathrm{s}.\mathrm{t}.\mathbf{T}\mathbf{1}=\mu ,{\mathbf{T}}^{⊺}\mathbf{1}=\nu ,\mathbf{T}(i,k)\ge 0.\hfill \end{array}$$

(16)

The pseudo-code of unsupervised GW learning is shown in Algorithm 3. We first initialize ${\mathit{\beta }}^{\left(s\right)},{\mathit{\beta }}^{\left(t\right)}$, and $\mathbf{T}$ as uniform distribution and randomly initialize $\Theta$ (Line 1). For each iteration (Line 2), we compute the intra-graph cost (Lines 3) followed by the inter-graph cost (Line 4). Next, the normalization trick is applied (Line 5). We then invoke [31,36] to learn $\{{\mathit{\beta }}^{\left(s\right)},{\mathit{\beta }}^{\left(t\right)},\Theta \}$ and $\mathbf{T}$ (Line 6).

Algorithm 3: Unsuper-GW.

We demonstrate that with the newly added parameters $\Theta$ in the attribution interaction step, under mild assumptions, Algorithm 3 can still converge. We have the following theorem, which is extended from the theoretical results of [14].

Theorem 2.

Let $F(\mathbf{T},\mathit{\beta },\Theta )$ denote the learning objective in Equation (16), with $\mathit{\beta }=({\mathit{\beta }}^{\left(s\right)},{\mathit{\beta }}^{\left(t\right)})$. Suppose that $0<{\tau }_T<{\displaystyle \frac{1}{L_f^T}},0<{\tau }_{\beta }<{\displaystyle \frac{1}{L_f^{\beta }}}$, and $0<{\tau }_{\Theta }<{\displaystyle \frac{1}{L_f^{\Theta }}}$, where $L_f^T$, $L_f^{\beta }$, and $L_f^{\Theta }$ are the gradient Lipschitz continuous modulus of $F(\mathbf{T},\mathit{\beta },\Theta )$, respectively. Then, with the condition in Equations (18)–(20), any limit point of the sequence ${\left\{({\mathbf{T}}^{\left(i\right)},{\mathit{\beta }}^{\left(i\right)},{\Theta }^{\left(i\right)})\right\}}_{i\ge 0}$ converges to a critical point of $\overline{F}(\mathbf{T},\mathit{\beta },\Theta )$, with

$$\overline{F}(\mathbf{T},\mathit{\beta },\Theta )=F(\mathbf{T},\mathit{\beta },\Theta )+{\mathbb{I}}_{\mathcal{T}}\left(\mathbf{T}\right)+{\mathbb{I}}_{\mathcal{B}}\left(\mathit{\beta }\right)+{\mathbb{I}}_{\mathcal{W}}(\Theta ),$$

(17)

where $\mathbb{I}(·)$ denotes the indicator function. Specifically, we have

$$\begin{array}{cc}\hfill \mathcal{T}& =\{\mathbf{T}\ge 0:\mathbf{T}\mathbf{1}=\mu ,{\mathbf{T}}^{⊺}\mathbf{1}=\nu \},\hfill \end{array}$$

(18)

$$\begin{array}{cc}\hfill \mathcal{B}& =\{({\mathit{\beta }}^{\left(s\right)},{\mathit{\beta }}^{\left(t\right)})\ge 0:\sum _{i=1}^3{\beta }_i^{\left(p\right)}=1,p=s,t\},\hfill \end{array}$$

(19)

$$\begin{array}{cc}\hfill \mathcal{W}& =\{\Theta \ge 0:\sum _{i=1}^{(K+1)d}{\Theta }_{ij}=1,j\in [1,d]\}.\hfill \end{array}$$

(20)

Proof. 

Our proof generally follows that of Theorem 5 in [14]. Note that by our definition, $\mathcal{T},\mathcal{B}$, and $\mathcal{W}$ are bounded sets, and $F(\mathbf{T},\mathit{\beta },\Theta )$ is a bi-quadratic function with respect to $\mathbf{T}$, $\mathit{\beta }$, and $\Theta$. To guarantee that $\Theta$ satisfies the above constraint, in each iteration, it is sufficient to apply an activation (e.g., ReLU) function followed by column-wise normalization. The proof then follows Theorem 5 in [14], where we can compute the gradient of F w.r.t. $\mathbf{T}$, $\mathit{\beta }$, and $\Theta$ to update the parameters with provable convergence.    □

• Relaxing the constraints on $\mathit{\beta },\Theta$. Through our empirical study, we find that the constraints on $\mathit{\beta }$ (as in [14]) and $\Theta$ (see Theorem 2) can be relaxed. That is, we do not force the non-negativity of parameters nor constrain that the (column-wise) summation equals 1. Please note that the theoretical analysis only guarantees convergence of the algorithm to a critical point, which aligns with the existing studies [14,15].

3.4. Self-Training Phase

We illustrate the self-training phase in Figure 4. It includes a pseudo-anchor selection step followed by the similarity-based cost propagation (SCP) procedure, which is used to enhance the state-of-the-art supervised method [12] for graph alignment.

3.4.1. Pseudo-Anchor Selection

• Motivation. As mentioned above, when using Gromov–Wasserstein distance (GWD) to model the graph alignment problem, $C_p$ represents the intra-graph cost, $C_p\in {\mathbb{R}}^{n\times n},p=s,t$, which is crucial for determining the matching accuracy [14], as it captures the relationships between nodes within a single graph. In this context, for a graph with n nodes, we can categorize the nodes into those that are easy to match and those that are more challenging to align. A natural question that arises is how to define which nodes are more likely to be matched accurately. Since $C_p$ represents the dependencies between nodes, a fundamental metric node degree can provide insight. Nodes with higher degree can access more information across the graph, capturing dependencies with multiple other nodes, which aligns well with $C_p^{\prime }$ role in representing overall intra-graph relationships. Conversely, nodes with a lower degree are relatively less capable of modeling inter-node relationships, making them harder to align accurately.

Based on the above analysis, we employ various OT-based methods [14,15] across different datasets to examine the node-matching process in detail. We observe that nodes with higher degrees are matched more accurately, which is consistent with our analysis. Take SLOTAlign [14] and two widely used datasets, Douban Online–Offline [33] and ACM-DBLP [37], as an example. As shown in Figure 5, we place nodes into bins corresponding to degrees in $[{\theta }^i,{\theta }^{i+1})$ for $i=0,\dots ,{\log }_{\theta }{d}_{max}$. We set $\theta =1.5$ to scatter the nodes so that they do not gather in few bins (we also remove bins that only contain less than 1% of all nodes). The red (resp. blue) bars denote the number of correctly aligned nodes in $G_s$ (resp. $G_t$) for each bin, while the red (resp. blue) lines show the corresponding prediction accuracy (i.e., Hits@1). It is clear that nodes with top degrees (e.g., top-10% to 30% nodes) have a significantly higher accuracy than the remaining nodes. Moreover, we also note that these nodes in $G_s$ (resp. $G_t$) tend to have large values of ${max}_k\mathbf{T}(i,k)$ (resp. ${max}_i\mathbf{T}(i,k)$), namely, the values in the i-th row (resp. k-th column) are more concentrated given that their summations are fixed.

The above observation motivates us to present a self-training strategy as follows. First, a set of matched nodes with high accuracy is selected as the pseudo-anchors. Second, we use a supervised graph alignment solution modified by our SCP approach, leveraging the pseudo-anchors for supervision to refine the input alignment matrix $\mathbf{T}$.

To be more precise, given the alignment matrix $\mathbf{T}$ obtained in the unsupervised learning phase and a predefined pseudo-anchors ratio defined by the hyperparameter q, we sort the value of $\left\{(u_i,{max}_k\mathbf{T}(i,k))\right|u_i\in G_s\}$ by descending the order of ${max}_k\mathbf{T}(i,k)$ and select the top-q node pairs $(u_i,M\left(u_i\right))$ as pseudo-anchors (it is also fine to first select nodes $v_k$ in $G_t$ and then find their matching nodes in $G_s$). In practice, we usually set $q\in [0.1,0.3]$.

3.4.2. Similarity-Based Cost Propagation (SCP)

During the self-training phase, we enhance the existing OT-based method, PARROT [12], to serve as the supervised model. We find that the positional embedding and the inter-graph cost computation of [12] is suboptimal (see Equations (8) and (9)). To be specific, for any two nodes $u_i\in G_s$ and $v_k\in G_t$ with close positional embeddings (i.e., ${\mathbf{R}}_s\left(i\right)\approx {\mathbf{R}}_t\left(k\right)$) or similar features (i.e., ${\mathbf{X}}_s\left(i\right)\approx {\mathbf{X}}_t\left(k\right)$), the value of ${\mathbf{C}}_{base}(i,k)$ is small. In other words, ${\mathbf{C}}_{base}$ denotes the node-wise distances across graphs, where matrix components with small values carry more information (and they are few in numbers according to the sparse nature of graph alignment). However, as Equation (9) essentially represents the RWR-based node similarity propagation on the product graph $G_s\otimes G_t$ in which ${\mathbf{C}}_{base}$ is regarded as the personalized vector [34], these small-valued components have minor effects in propagation. Particularly, $vec\left({\mathbf{C}}_{base}\right)$ can be decomposed into

$$vec\left({\mathbf{C}}_{base}\right)={\mathbf{C}}_{base}(1,1)·{\mathbf{e}}_1+\dots +{\mathbf{C}}_{base}(n_1,n_2)·{\mathbf{e}}_{n_1{n}_2},$$

(21)

where ${\mathbf{C}}_{base}(i,k)$ denotes the component at i-th row and k-th column, and ${\mathbf{e}}_s$ is an $n_1{n}_2$-dimensional one-hot vector with the s-th element being 1. It is clear that similar node pairs with small ${\mathbf{C}}_{base}(i,k)$s contribute less to the propagation results ${\mathbf{C}}_{rwr}$.

Instead, we adopt a node-wise similarity-based approach to conduct the propagation, which is referred to as similarity-based cost propagation (SCP). We first define the node-wise similarity rather than the distances as the base case:

$${\mathbf{S}}_{base}=\alpha e^{\tau ({\mathbf{R}}_s{\mathbf{R}}_t^{⊺}-1)}+(1-\alpha )e^{\tau ({\mathbf{X}}_s{\mathbf{X}}_t^{⊺}-1)}.$$

(22)

where $\tau >0$ is a temperature parameter. Note that for normalized ${\mathbf{R}}_s\left(i\right),{\mathbf{R}}_t\left(k\right),{\mathbf{X}}_s\left(i\right)$, and ${\mathbf{X}}_t\left(k\right)$, we have ${\mathbf{S}}_{base}(i,k)\in [e^{-2\tau },1]$. Next, the inter-graph cost with supervision, ${\mathbf{C}}_{int}$ is computed as Equation (23):

$$\begin{array}{c}\hfill vec\left({\mathbf{C}}_{int}\right)=(1-\delta )\gamma {\mathbf{W}}_{s\otimes t}vec\left({\mathbf{C}}_{int}\right)+(1+\delta )({\mathbf{1}}_{n_1{n}_2}-vec\left({\mathbf{S}}_{base}\right)).\end{array}$$

(23)

We elaborate on the meaning of ${\mathbf{C}}_{int}$ as Equation (24). According to the definition of RWR (i.e., PPR [34]), we have

$$\begin{array}{cc}\hfill \mathrm{vec}\left({\mathbf{C}}_{int}\right)& ={(\mathbf{I}-(1-\delta )\gamma {\mathbf{W}}_{s\otimes t})}^{-1}(1+\delta )\times ({\mathbf{1}}_{n_1{n}_2}-\mathrm{vec}\left({\mathbf{S}}_{base}\right))\hfill \\ \hfill & =\sum _{i=0}^{\infty }(1+\delta ){(1-\delta )}^i{\gamma }^i{\mathbf{W}}_{s\otimes t}^i{\mathbf{1}}_{n_1{n}_2}-\sum _{i=0}^{\infty }(1+\delta ){(1-\delta )}^i{\gamma }^i{\mathbf{W}}_{s\otimes t}^i\mathrm{vec}\left({\mathbf{S}}_{base}\right).\hfill \end{array}$$

(24)

Note that the second term of Equation (24) is the RWR propagation with personalized vector $vec\left({\mathbf{S}}_{base}\right)$. Therefore, it corresponds to node-wise similarity propagation on $G_s\otimes G_t$. To transform the similarities into distances, we subtract it from the first term, which refers to the propagation with an all-one vector. Another choice is to compute the inter-graph similarities ${\mathbf{S}}_{rwr}$, which equals to the second term of Equation (24). Next, we compute ${\mathbf{C}}_{int}^{\prime }=max({\mathbf{1}}_{n_1}{\mathbf{1}}_{n_2}^{⊺}-{\mathbf{S}}_{rwr},{\mathbf{0}}_{n_1\times n_2})$. This gives comparable or slightly inferior results compared to Equation (23). If it holds that ${\mathbf{S}}_{base}(i,k)\le 1$ for all i and k, we have ${\mathbf{C}}_{int}\ge {\mathbf{0}}_{n_1\times n_2}$.

3.4.3. Learning Algorithm

As shown in Algorithm 4, we first conduct pseudo-anchor selection (Line 1), followed by similarity-based cost propagation (Line 2). We then conduct supervised OT-based learning, for example, by invoking [12] with the refined inter-cost (Line 3), and use the refined alignment matrix ${\mathbf{T}}^{\prime }$ to update our prediction (Line 4), including the pseudo-anchor nodes.

• Remark. In principle, our self-training process can be used to improve any existing OT-based unsupervised methods such as [14,15].

Algorithm 4: Self-training.

Input: The alignment matrix $\mathbf{T}$ obtained from the unsupervised step, the ratio of pseudo-anchors q Output: The refined alignment matrix ${\mathbf{T}}^{\prime }$ Sort $\left\{(u_i,{max}_k\mathbf{T}(i,k))\right|u_i\in G_s\}$ by descending order of ${max}_k\mathbf{T}(i,k)$ and select the top-$q\%$ node pairs as pseudo-anchors; Compute the node-wise similarities (Equation (22)) and the inter-graph cost (Equation (23)); Replacing the original transport cost in PARROT [12] with SCP to obtain a refined alignment matrix ${\mathbf{T}}^{\prime }$; return ${\mathbf{T}}^{\prime }$;

3.5. Complexity Analysis

We present a concise complexity analysis of PORTRAIT. Consider source graph ${\mathcal{G}}_s$ has $n_s$ nodes and $m_s$ edges while target graph ${\mathcal{G}}_t$ has $n_t$ nodes and $m_t$ edges, and the feature dimension is d. Defining $n=max(n_s,n_t)$, the computational requirements are as follows:

Intra-graph Cost Computation: The construction of cost matrices ${\mathbf{C}}_s$ and ${\mathbf{C}}_t$ in Equation (13) requires $O\left(n^{2}d\right)$ operations, as established in [14,15]. Evaluating the GW term in Equation (6) dominates the complexity with $O\left(n^3\right)$ operations. This complexity governs both the $\mathit{\beta }$-update and $\mathbf{T}$-update steps in Equation (16). Given the typical condition $d\ll min(n_s,n_t)$, the total complexity reduces to $O\left(n^3\right)$, maintaining parity with existing optimal transport-based alignment methods [14,15,32].

Note that the computational bottleneck originates from the Gromov–Wasserstein learning process, which employs proximal gradient methods [31,36] for alignment matrix updates. While this complexity is inherent to current OT-based graph alignment approaches, we recognize that scalability remains an important unresolved challenge that will be prioritized in our future research.

4. Experimental Analysis

4.1. Experimental Setup

• Datasets: Three widely-used datasets for attributed graph alignment [9,12,14,17] are adopted, which contain information on graph structures and node features (see

  Table 2. Datasets and their statistics.

  Dataset	$\mathit{n}=\left|\mathit{V}\right|$	$\mathit{m}=\left|\mathit{E}\right|$	Features	Answers
  Douban [33]	1118	3022	538	1118
  	3906	$\mathrm{16,328}$
  ACM-DBLP [37]	9872	$\mathrm{39,561}$	17	6325
  	9916	$\mathrm{44,808}$
  PPI [38]	1767	$\mathrm{16,159}$	50	1767
  	1767	$\mathrm{16,159}$
  Allmv-Imdb [22]	5713	$\mathrm{119,073}$	14	5174
  	6011	$\mathrm{124,709}$

  ).

Douban Online–Offline [33] contains two social networks corresponding to an online graph and an offline graph of Douban users. Node features denote user locations. The larger online graph contains all the users in the offline graph.

The ACM-DBLP [37] dataset contains two academic collaboration networks extracted from ACM and DBLP, respectively. Nodes represent authors while edges represent collaborations. The numbers of papers published in venues (of four research areas) are used as node features.

The Allmv-Imdb [22] dataset contain two networks. The Allmv network is derived from the Rotten Tomatoes website, where two films are connected by an edge if they share at least one common actor. The Imdb network is constructed from the Imdb website using a similar approach. The alignment indicates film identity, incorporating 5174 anchor links.

PPI [38] is a semi-synthetic dataset where two different node permutations of a protein interaction network are generated. We mainly use it to demonstrate the important properties of OT-based approaches in an adversarial setting where the motivation of self-training is less obvious. Note that this dataset is much easier than the two real datasets as all baselines achieve high accuracy.

• Baselines: We include representative embedding-based methods WAlign [17], GAlign [22], and GTCAlign [23], and use GWL and SLOTAlign as the OT-based competitors because they achieve state-of-the-art performance on the aforementioned datasets according to [14,15]. We do not include [32,39,40] as baselines since they are significantly outperformed by GWL and SLOTAlign [14] and thus are not comparable with ours. We also omit [29,30] as they are limited to graphs with no larger than one hundred nodes. For all baselines, we run the code provided by the authors with the default configuration.
• Evaluation Metrics: We evaluate the performance of all models using the well-adopted metric Hits@k with $k\in \{1,5,10\}$. Hits@k indicates whether the ground truth correspondence of a node occurs in the top-k predictions ordered by the alignment probability. In particular, we put emphasis on Hits@1 because it reflects the percentage of matching that is correctly found.
• Hyperparameter Settings: For PORTRAIT, the learning rate of $\mathit{\beta }$ is the same as SLOTAlign [14], while the learning rate of $\Theta$ is set to 0.01. In the self-training phase, we fix the ratio of pseudo-anchors q as $0.2$, $\alpha$ (in Equation (7)) as 0.1, $\delta$ as $0.15$, and $\gamma$ as $0.2$, following [12]. We set the number of layers (i.e., K) of the attribute propagation step as 2. For the similarity-based cost propagation, we set the temperature parameter $\tau$ to 10.

All experiments are conducted on a high-performance computing server with an Intel13 13900KF CPU and a GV-4090 GAMINGOC-24GD GPU.

4.2. Evaluation of Model Accuracy

We first evaluate the accuracy of unsupervised graph alignment methods. As shown in

Table 3. Evaluation of the model accuracy. We mark the best and second-best results of all unsupervised model variants with bold font and underline, respectively.

	Douban Online–Offline			ACM-DBLP			PPI			Allmv-Imdb
Model	Hits@1	Hits@5	Hits@10	Hits@1	Hits@5	Hits@10	Hits@1	Hits@5	Hits@10	Hits@1	Hits@5	Hits@10
WAlign	39.45	62.25	71.47	63.43	83.18	86.58	88.51	92.98	93.87	52.59	70.94	76.53
GAlign	45.24	67.77	78.13	70.17	86.98	91.25	89.07	90.28	94.04	82.14	86.35	90.03
GTCAlign	60.91	76.77	82.18	60.83	75.52	80.11	89.25	92.81	94.07	84.73	89.92	91.37
GWL	3.49	8.32	9.93	56.36	77.09	82.18	86.76	88.06	88.62	87.82	92.31	93.83
GWL + ST	10.55	23.61	30.59	64.18	85.31	89.69	89.24	93.09	93.99	90.12	93.53	94.07
SLOTAlign	51.43	73.43	77.73	65.63	85.84	87.76	89.30	92.53	93.49	90.60	92.75	93.12
SLOTAlign + ST	55.18	78.03	82.86	69.88	88.62	91.28	89.36	93.01	94.03	92.07	93.81	94.26
PORTRAIT	63.32	86.40	90.87	73.45	90.13	93.50	89.75	93.04	94.17	95.33	96.27	96.72
PORTRAIT w/o ST	58.68	78.35	83.54	69.05	87.04	90.61	89.36	92.64	93.66	93.75	94.79	95.26
PORTRAIT w/o ST+$\Theta$	50.45	68.25	72.90	66.22	84.96	87.32	89.25	92.30	93.06	89.98	93.01	93.79
PORTRAIT w/o ST+Norm	45.61	63.95	68.92	63.54	81.14	84.62	85.74	89.59	90.72	87.96	92.77	93.91

, consistent with [14], SLOTAlign outperforms GWL while the latter is not stable on the Douban dataset, whereas WAlign and GAlign, the representative embedding-based methods, are surpassed by OT-based counterparts. Generally speaking, PORTRAIT achieves the best accuracy among all unsupervised model variants on the four datasets, proving the effectiveness of our model. We omit the standard errors following [14,15,17] because the prediction accuracy is very concentrated for all compared methods. Specifically, with all stages included, our proposed model outperforms the state-of-the-art method (i.e., SLOTAlign [14]) in terms of Hits@1 by 23.12% on the Douban dataset, and by 11.91% on the ACM-DBLP dataset. For the other evaluation metrics, such as Hits@5 and Hits@10, PORTRAIT similarly outperforms the other baselines. On the PPI dataset, all methods have comparable performance.

Next, we analyze our self-training strategy. Recall that the self-training stage is independent of unsupervised GW learning and can be integrated to improve existing OT-based solutions. As demonstrated in Table 3 where ST denotes the self-training phase, the performance of GWL, SLOTAlign, and PORTRAIT are all improved by a significant margin on Douban and ACM-DBLP. Specifically, with self-training, GWL, SLOTAlign, and PORTRAIT achieve the accuracy improvement with respect to Hits@1 by 202.29%, 7.29%, and 7.91% on the Douban dataset, and 13.88%, 6.48%, and 6.37% on the ACM-DBLP dataset, respectively.

• Ablation Study: Different variants of PORTRAIT are evaluated in terms of prediction accuracy (see Table 3), where the self-training phase (denoted as ST), the attributed interaction stage (denoted as $\Theta$), and cost normalization (denoted as Norm) are deactivated. For PORTRAIT w/o ST + Norm, we train it following a similar procedure with SLOTAlign [14]. Particularly, we first train $\{{\mathit{\beta }}^{\left(s\right)},{\mathit{\beta }}^{\left(t\right)},\Theta \}$ and $\mathbf{T}$ alternatively and then solely update $\mathbf{T}$ with hundreds of iterations. It can be concluded that all three modules contribute significant accuracy improvement on Douban and ACM-DBLP while helping our model to achieve the best performance on PPI. It also indicates the necessity of attribute interaction in improving model capability.

We also conduct an extensive evaluation of the self-training strategy (see

Table 4. Evaluation of our self-training strategy.

	Douban Online–Offline			ACM-DBLP			PPI			Allmv-Imdb
Model	Hits@1	Hits@5	Hits@10	Hits@1	Hits@5	Hits@10	Hits@1	Hits@5	Hits@10	Hits@1	Hits@5	Hits@10
PORTRAIT	63.42	86.40	90.87	73.45	90.13	93.50	89.75	93.04	94.17	95.33	96.27	96.72
PORTRAIT w/degree-based	62.25	86.25	90.78	72.66	89.66	93.35	89.58	93.03	94.13	95.12	96.18	96.39
PORTRAIT w/PARROT	58.32	81.92	86.35	70.68	89.16	92.88	89.60	92.92	94.11	94.33	95.29	95.61

[1] PORTRAIT includes the self-training phase.

), including the default approach which chooses pseudo-anchors according to alignment probability, the degree-based heuristic that uses the predicted matchings of top-degree nodes as supervision, and a model variant that replaces SCP with the position-aware transport cost in [12]. The degree-based heuristic achieves nearly identical accuracy as the default approach, which verifies our motivation that these nodes have larger values of alignment probability. In contrast, the original solution in [12] demonstrates inferior performance. This result validates the effectiveness of our SCP module.

• Robustness Analysis: To validate the robustness of our proposed model, we conduct comprehensive experiments involving edge perturbations (ranging from 10% to 40% noise ratios) across three real-world datasets. For comparative analysis, we selected two representative baseline methods: GAlign as the embedding-based approach, and SLOTAlign as the optimal transport-based method. As illustrated in Figure 6, our model demonstrates consistently superior robustness across all noise levels, exhibiting significantly smaller performance degradation compared to baseline methods. Particularly noteworthy is the observation that at 30% noise contamination, a substantial perturbation level, our model maintains prediction accuracy comparable to that achieved by competing methods under noise-free conditions.
• Parameter Sensitivity: We study the effects of hyperparameters on model accuracy, including the number of layers K in the attribute propagation step, the ratio of the pseudo-anchors, and the temperature parameter in the SCP module on the two real datasets. We use Hits@1 as the evaluation metric. The results are presented in Figure 7. By analyzing the impact of attribute propagation layers, a subtle decline in performance on the Douban dataset is noticeable with the increasing value of K. We speculate that this decline may be attributed to a certain degree of oversmoothing, potentially caused by the heightened number of layers. Interestingly, as the pseudo-anchor ratio increases, the model’s performance initially improves, followed by a slight decline. We attribute this phenomenon to higher confidence (92%) in pseudo-anchors within a smaller range; as the ratio grows, additional noise is introduced, which diminishes performance. Meanwhile, we observe a slight improvement in performance on the ACM-DBLP dataset as the temperature coefficient increases. In general, our model exhibits consistent robustness to variations of hyperparameters, highlighting the stability of our proposed approach.
• Efficiency Comparison: To evaluate the computational efficiency of our model, we measure the runtime to convergence across three datasets. As shown by Figure 8 and Figure 9, our framework achieves significantly faster convergence on real-world datasets compared to baseline methods. This advantage becomes particularly pronounced on larger-scale datasets (e.g., ACM-DBLP), which we attribute to the model’s enhanced expressive power, which enables faster convergence with fewer epochs, and the matrix multiplication in Gromov–Wasserstein learning process could fully leverage the computing power of GPUs. For the synthetic dataset PPI, all methods achieve comparable efficiency.

4.3. Properties of OT-Based Methods

To further validate our motivation, we conduct a more detailed analysis of the OT-based graph alignment process across different datasets, and investigate the correlation between node degrees and the row-wise (column-wise) maximum alignment probabilities.

Table 5. Correlation between node degree and ${max}_k\mathbf{T}(i,k)$ (resp. ${max}_i\mathbf{T}(i,k)$).

	Douban Online–Offline		ACM-DBLP
Set of Nodes	${\mathbf{G}}_{\mathit{s}}$	${\mathbf{G}}_{\mathit{t}}$	${\mathbf{G}}_{\mathit{s}}$	${\mathbf{G}}_{\mathit{t}}$
Top 10%	$2.26\times {10}^{-4}$	$2.36\times {10}^{-4}$	$8.63\times {10}^{-5}$	$8.23\times {10}^{-5}$
Top 30%	$1.94\times {10}^{-4}$	$1.94\times {10}^{-4}$	$7.79\times {10}^{-5}$	$7.83\times {10}^{-5}$
All	$1.28\times {10}^{-4}$	$1.11\times {10}^{-4}$	$6.38\times {10}^{-5}$	$6.39\times {10}^{-5}$
Last 70%	$9.97\times {10}^{-5}$	$7.61\times {10}^{-5}$	$5.77\times {10}^{-5}$	$5.77\times {10}^{-5}$

shows the average values for top-10% to 30% and the remaining nodes as well as the entire node set. It is clear that a strong correlation exists between the degree of a node and the predicted alignment probability, leading to higher accuracy. Meanwhile, we show the relationship between node degree and prediction accuracy (i.e., Hits@1) during the learning process. For ACM-DBLP, Douban, and PPI, we demonstrate the results of GWL (Figure 10a–d, Figure 11a–d, and Figure 12a–d), SLOTAlign (Figure 10e–h, Figure 11e–h, and Figure 12e–h), and PORTRAIT (Figure 10i–l, Figure 11i–l, and Figure 12i–l), respectively. It is obvious that nodes of large degrees are first aligned. Moreover, as the time costs per epoch for different OT-based methods are close, we conclude that PORTRAIT converges much faster than the other two baselines (also see Figure 9).

5. Other Related Work

Existing works for graph alignment can be broadly categorized according to the problem inputs. Graph structure-based methods [7,15,19,20,41,42,43,44,45] only utilize topological information for alignment, while our model falls into the category of attributed graph alignment [3,9,12,14,33,39,46,47,48,49,50,51,52] where node attributes are also available. Another problem setting is knowledge graph (KG) alignment (e.g., [10,40,53]), and comprehensive surveys are referred to [54,55].

From the perspective of learning paradigm, consistency-based methods [9,12,56,57,58] are widely used for structure-only and attributed graph alignments. Embedding-based solutions [13,18,19,20,21] follow the idea of “Embed-Then-Cross-Compare” while GNNs are recently employed [17,40] and cross-graph propagation is considered [20]. Another line of work adopts adversarial learning [17,47]. Pei et al. [59] propose a curriculum learning approach, but they focus on the supervised setting with noise while we aim at OT-based methods without supervision. Similarly, [43] handles the degree differences in the embedding space of KG alignment for the semi-supervised setting. We also have witnessed a line of algorithms for efficient OT computation [60,61,62], which might accelerate graph alignment in the near future.

6. Conclusions and Limitation

We propose PORTRAIT, an unsupervised approach that enhances optimal transport-based graph alignment by leveraging attribute interactions and self-training. Specifically, we design flexible modules with enhanced expressive power to effectively integrate richer structural and attribute information, while introducing a cost normalization module to improve both accuracy and stability. Then, by selecting high-confidence matching nodes as pseudo-labels, we present a self-training strategy with our refined cost computation process to significantly enhance the accuracy of our model and other existing OT-based methods. Extensive experimental results demonstrate our model’s superior performance, achieving a 5% improvement in Hits@1, which effectively validates our motivations. Although the proposed method achieves outstanding prediction accuracy, scalability remains a challenge that will constitute the primary focus of our future research.