Spatial-Temporal Attention TCN-Based Link Prediction for Opportunistic Network

Abstract

Link prediction for opportunistic networks faces the challenges of frequent changes in topology and complex and variable spatial-temporal information. Most existing studies focus on temporal or spatial features, ignoring ample potential information. In order to better capture the spatial-temporal correlations in the evolution of networks and explore their potential information, a link prediction method based on spatial-temporal attention and temporal convolution network (STA-TCN) is proposed. It slices opportunistic networks into discrete network snapshots. A state matrix based on topology information and attribute information is constructed to represent snapshots. Time convolutional networks and spatial-temporal attention mechanisms are employed to learn spatial-temporal information. Furthermore, to better improve link prediction performance, the proposed method converts the auto-correlation error into non-correlation error. On three real opportunistic network datasets, ITC, MIT, and Infocom06, experimental results demonstrate the superior predictive performance of the proposed method compared to baseline models, as shown by improved AUC and F1-score metrics.

1. Introduction

An opportunistic network [1,2] is a type of mobile self-organized network in which the source node is not directly connected to the target node, and the communication between the two nodes is realized by encounters through nodes’ movement, and the data is transmitted by way of “store-carry-forward”. Opportunistic networks have been used in many fields [3], such as socially oriented services, personal and environmental services, intelligent transportation systems, earthquake prediction [4], and so on. The goal of link prediction is to predict the future network topology based on historical information. Link prediction can offer support for the above applications, which is one of the hotspots in the current research of opportunistic networks. In this paper, we study opportunistic networks, whose links have two states: connected and disconnected.

The limitations of current link prediction methods can be summarized in the following three aspects:

• Most temporal models focus on capturing temporal features, neglecting the correlation between spatial and temporal features.
• Many nodes in real networks have rich attributes that change over time, and these attributes are intertwined in both the temporal and spatial dimensions [5]. It is challenging to determine which factor is more important. Although some methods consider both spatial and temporal features, they fail to discriminate the importance of different factors.
• Existing studies do not deal with auto-correlation errors. When networks are sliced into snapshots, in the process of collecting and modeling time series, auto-correlation errors will arise unavoidably, which will affect prediction performance.

To alleviate the aforementioned limitations, this paper proposes a link prediction model based on Spatial-Temporal Attention Mechanism and Temporal Convolution Networks (STA-TCN). A state matrix is constructed by combining the topology information, the link frequency, and the link duration of node pairs so that potential information can be learned better. The spatial-temporal attention mechanism and temporal convolution are integrated into the encoder-decoder to extract spatial-temporal information. Inspired by [6], the auto-correlation error in the predicting process is adjusted to the non-correlation error to improve the link prediction performance. The main contributions of this paper are as follows:

• A novel network representing method is proposed to characterize networks by state matrix, which is constructed by the first-order adjacency matrix, quadratic adjacency matrix, and correlation matrix.
• A spatial-temporal dual-attention mechanism is employed to learn the node representations in both spatial and temporal aspects; each snapshot and each neighbor are obtained with different attention to achieve more accurate spatial-temporal characteristics.
• An error transformation method is proposed. The auto-correlation error in the STA-TCN model is converted into non-correlated error to further improve the prediction performance.

2. Related Work

Link prediction, as an important task in network data mining, has been widely studied and falls into the following categories: matrix factorization-based, probabilistic model-based, and machine learning-based approaches.

The matrix factorization-based methods decompose the adjacency matrix of a network into the product of low-rank matrices to capture the latent relationships between nodes. Ref. [7] proposed an architecture for link prediction using the NMF algorithm, which utilized different kernel functions to extract local and global features of the network. Ref. [8] introduced a graph-regularized non-negative matrix factorization method that captured the local information of snapshots through graph regularization and calculated the importance of nodes in each snapshot using the PageRank algorithm. Finally, a non-negative matrix factorization model is jointly optimized to preserve both local and global information. Ref. [9] proposed a robust graph regularized non-negative matrix factorization algorithm modeling the network topology and node attributes.

The probability model-based methods utilize principles of probability theory and mathematical statistics to infer unknown phenomena based on known data or assumptions. Ref. [10] presented a social network evolution analysis method based on triplets (SNEA), where the prediction algorithm learns the evolution of the triplet transition probability matrix. Ref. [11] proposed a deep generative model for graphs, which learns the likelihood of missing links in the graph using autoregressive generative models. Ref. [12] introduced an adaptive method to assess the likelihood of a link between nodes.

In recent years, deep learning-based link prediction models have gained increasing attention due to their ability to automatically discover features from data [13,14]. Ref. [15] proposed an embedding method called dyngraph2vec, which used multiple non-linear layers to learn structural patterns in each snapshot and utilized recurrent layers to learn temporal transformations in the network. To reduce the number of model parameters and improve learning efficiency, DynRNN and DynAERNN models were subsequently proposed. Ref. [16] introduced the Evolving Graph Convolutional Network (EvolveGCN), which evolved the graph convolutional parameter matrix using a Recurrent Neural Network (RNN) to capture the dynamics of graph sequences. Ref. [17] proposed an end-to-end link prediction method called Graph Convolution Network Embedded Long Short-Term Memory (GC-LSTM), which embedded Long Short-Term Memory (LSTM) into the graph convolutional network to learn the temporal features of all snapshots in a dynamic network. It used the graph convolutional network (GCN) to capture the local structural attributes of nodes and their relationships in each snapshot. Ref. [18] proposed a novel encoder-LSTM-decoder (E-LSTM-D) model to predict end-to-end dynamic links. The model integrated stacked LSTMs into the architecture of the encoder-decoder, learning not only low-dimensional representations and non-linearity but also the time dependencies between consecutive network snapshots. Ref. [19] replaced LSTM with Gated Recurrent Units (GRUs) in the E-LSTM-D model to obtain the E-GRUs-D model, which improved the training efficiency of the model while sacrificing a small amount of accuracy. Ref. [20] proposed a GRU prediction model combined with structural embedding and designed various loss constraints to improve prediction accuracy. Ref. [21] introduced node-adaptive parameter learning and data-adaptive generation modules to enhance traditional GCN and further proposed the Adaptive Graph Convolutional Recurrent Network (AGCRN) to automatically capture fine-grained spatial and temporal correlations in traffic sequences.

To sum up, the above methods mainly use graph neural networks combined with recurrent neural networks to capture topological and temporal information, which neglects the rich potential information of nodes and the interactions between spatial and temporal dimensions. Compared with the above work, the STA-TCN model proposed in this paper takes state matrix to represent the snapshot. A spatial-temporal attention mechanism is employed to better learn the spatial and temporal features. To further improve the network’s link prediction performance, the auto-correlation error is transformed into the non-correlation error.

3. Methodology

3.1. Problem Statement

Given a sequence of network snapshots of length T as $G=\{G_0,...,G_t,...,G_T\}$, $G_t$ denote a snapshot of the network at moment t, defined as $G_t=(V_t,E_t)$, where $V_t$ denotes the set of nodes and M is the total number of nodes, and $E_t$ denotes the set of links. Opportunistic network link prediction aims to predict $G_T$ based on a sequence of network snapshots of length T, as shown in Figure 1.

3.2. Representation of Opportunistic Network

The opportunistic network is an undirected network, which can be divided into discrete network snapshots by time slicing. For a specific snapshot at time t, a first-order adjacency matrix $A_t\in {ℝ}^{M\times M}$, quadratic adjacency matrix $B_t\in {ℝ}^{M\times M}$, node attribute matrix $X_t\in {ℝ}^{M\times d}$, and correlation matrix $C_t\in {ℝ}^{M\times M}$ are established. In this paper, a snapshot is represented by state matrix $S_t$, which is constructed by $A_t$, $B_t$ and $C_t$ as shown in Figure 2:

This section will provide a detailed explanation of the construction process for each of its components.

3.2.1. Topological Feature Representation

(1)

First-order adjacency matrix

The first order adjacency matrix $A_t$ represents the state of connected edges within a network snapshot t. If node $v_i$ and node $v_j$ are linked, then the element $a_{t;i,j}=1$, otherwise $a_{t;i,j}=0$.

(2)

Quadratic adjacency matrix

The quadratic adjacency matrix $B_t\in {ℝ}^{N\times N}$, captures the topological information of the network using the power of two of the first-order adjacency matrix and normalizes it to obtain the quadratic adjacency matrix $B_t$, which is defined as:

$$\begin{array}{c}{B}_t=Normal\left(A_t^2\right)\end{array}$$

(1)

3.2.2. Attribute Feature Representation

Opportunistic networks in which communication of nodes are frequently connected and interrupted. The link duration of node pairs indicates the duration of information transfer between nodes, and link frequency represents the strength of the relationship between nodes. The changes in link duration and link frequency can reveal the dynamic evolution of interaction patterns and relationships between nodes. Thus, the attribute vectors of the nodes are constructed based on the starting link moment $l{s}_t\left(i,j\right)$, the ending link moment $l{d}_t\left(i,j\right)$ and link frequency $l{c}_t\left(i,j\right)$.

When the majority of node pairs within a snapshot have a link duration greater than the slice size, the impact of the number of link frequency on the calculation of node attribute features is minimal, indicating that the magnitude of the link frequency can be disregarded. In that case, the node attribute feature vector $x_t\left(i,j\right)$ is shown as:

$$x_t\left(i,j\right)=\frac{l{d}_t\left(i,j\right)-l{s}_t\left(i,j\right)}{T}$$

(2)

where the i-th row of $x_t\left(i,j\right)$ denotes the attribute vector of node $v_i$ in the t-th snapshot.

When the majority of node pairs within a snapshot have a link duration less than the slice size, indicating that multiple links emerge in a snapshot, it is necessary to consider link frequency in the calculation of node attribute feature vector. The node attribute feature vector can be defined as:

$$x_t\left(i,j\right)=\frac{{\displaystyle \sum \left(l{d}_t\left(i,j\right)-l{s}_t\left(i,j\right)\right)}}{l{c}_t\left(i,j\right)\ast T}$$

(3)

With the obtained node attribute vectors, we can construct $C_t\in {ℝ}^{N\times N}$, the time-specific correlation matrix [22], which aims to capture the attribute correlation between nodes within a network snapshot t. Specifically, for a snapshot t, $C_{t;ij}$ captures the extent to which node $v_i$ is affected by node $v_j$. The correlation between the attribute vectors of node $v_i$ and the attribute vectors of node $v_j$ is captured by a normalization function, as shown by Formula (4):

$$\begin{array}{c}{C}_{t;ij}=\frac{e^{\theta {\left(x_t^i\right)}^T\varphi \left(x_t^j\right)}}{{{\displaystyle \sum }}_j^N{e}^{\theta {\left(x_t^i\right)}^T\varphi \left(x_t^j\right)}}\end{array}$$

(4)

where $\theta (·)$ and $\varphi (·)$ are embedding functions.

3.2.3. State Matrix Construction

State matrix $S_t$ is employed to represent a snapshot, which consists of the topological information of the network and the node attribute vectors, shown as following:

$$\begin{array}{c}{S}_t={\lambda }_A{A}_t+{\lambda }_B{B}_t+{\lambda }_C{C}_t\end{array}$$

(5)

where $A_t$ is the first order adjacency matrix, $B_t$ is the quadratic adjacency matrix, and $C_t$ is the correlation matrix. The matrices $A_t$, $B_t$ and $C_t$ are linearly combined with the learnable parameters ${\lambda }_A$, ${\lambda }_B$ and ${\lambda }_C$.

3.3. Link Prediction Model

The model uses the state matrix sequence as input. It encodes the sequence using spatial attention and TCN to obtain feature vectors. It decodes feature vectors using temporal attention and TCN to obtain the node’s feature matrix. Finally, it takes a fully connected layer to output the predicted matrix.

3.3.1. Overall Framework

In this paper, we propose a link prediction model STA-TCN based on temporal convolutional network and attention mechanism, as shown in Figure 3. The architecture of STA-TCN is composed of two components: a spatial attention-based encoder and a temporal attention-based decoder. The encoder extracts the spatial features [23] to obtain the matrix sequence $\left\{{\tilde{X}}_0,...,{\tilde{X}}_t,...,{\tilde{X}}_{T-1}\right\}$, and inputs the sequence into the TCN [24] for further encoding to obtain the spatial-temporal features $\left\{H_0,...,H_t,...,H_{T-1}\right\}$; the decoder introduces the temporal attention to process the encoded features to obtain $\left\{Q_0,...,Q_t,...,Q_{T-1}\right\}$, and inputs the processed features into the TCN for further decoding to get the node representation ${\tilde{H}}_T$, and go through the fully connected layer to get the predict network for the next moment ${\widehat{A}}_T$.

3.3.2. Spatial Attention-Based Encoder

The input to the encoder is the state matrix $S_t$. As different neighbors have varying impacts on the node, the attention mechanism is employed to dynamically capture the most important neighbors. To accomplish this, a spatial attention module is designed to identify the significant neighbors of a node, as illustrated in Figure 4. The spatial attention mechanism is defined by the following:

$$\begin{array}{c}{u}_t^i=W_{1}tanh\left(\left(W_2{\tilde{a}}_t^i\right){\left({\tilde{a}}_t^i{W}_3\right)}^T+b_1\right)\end{array}$$

(6)

where $W_1$, $W_2$ and $W_3$ are the learnable parameter matrices, ${\tilde{a}}_t^i$ denotes the i-th row vector of ${\tilde{A}}_t$, $b_1$ is the bias vector, and $u_t^i$ denotes the attention value of the i-th node at the network snapshot t. The attention weight ${\alpha }_t^i$ of the normalized node $v_i$ at moment t is shown as follows:

$${\alpha }_t^i=\frac{\exp \left(u_t^i\right)}{{{\displaystyle \sum }}_{j=1}^{N}exp\left(u_t^i\right)}$$

(7)

The node representation vector ${\tilde{x}}_t^i$ acquired by spatial attention is shown as following:

$$\begin{array}{c} {\tilde{x}}_t^i={\alpha }_t^i\otimes {\tilde{a}}_t^i+{\tilde{a}}_t^i\end{array}$$

(8)

where $\otimes$ denotes the product-by-element operation.

The series $\left\{{\tilde{X}}_0,\dots ,{\tilde{X}}_t,\dots ,{\tilde{X}}_{T-1}\right\}$ weighted by spatial attention is input into the time-series model TCN, which is a hierarchical model composed of multiple layers of dilated causal convolutions for extracting relationships. Figure 5 illustrates the architecture of a TCN with 3 layers of dilated causal convolutions, where the output of the final layer captures the temporal dynamics in the history. Using a TCN with L layers to establish the temporal model of the node $v_i$, where L represents the number of convolutional layers, each layer includes dilated convolutions, activate functions and normalization operations. Node $v_i$ at time t uses dilated causal convolutions for computation as shown as:

$$\begin{array}{c}{\mathrm{x}}^i{\star }_{t}F={\displaystyle {\displaystyle \sum }_{k=1}^K}{f}_k{\tilde{x}}_{t-d\left(K-k\right)}^i\end{array}$$

(9)

where ${\star }_t$ denotes the convolution operation at time t, f denotes the filter, K denotes the filter size, and d is the dilating factor.

3.3.3. Temporal Attention-Based Decoder

The decoder decodes the spatial-temporal vectors generated by the encoder to obtain the predicted adjacency matrix of the network at the next moment, thereby completing the link prediction. Since different snapshots provide different amounts of information, a temporal attention module is designed to automatically assign different weights to each snapshot, as shown in Figure 6. To automatically obtain different degrees of attention values for different network snapshots. The temporal attention mechanism calculates the attention weight for each snapshot using the follows:

$$g_t=W_4\tanh \left(\left(W_5{h}_t\right){\left(h_t{W}_6\right)}^T+b_2\right)$$

(10)

$${\beta }_t=\frac{\left|g_t\right|}{{{\displaystyle \sum }}_{m=0}^{T-1}\left|g_m\right|}$$

(11)

where $W_4$, $W_5$ and $W_6$ are the learnable parameter matrices, $h_t$ denotes the row vector of spatial-temporal feature $H_t$ at network snapshot t, $b_2$ is the bias vector, and $g_t$ denotes the attentional value at network snapshot t. The normalized ${\beta }_t$ denotes the attention weight at time t. The weighted summation with the encoder spatial-temporal features yields the node representation vector $q_t$, which is input to the decoder TCN model, and the node representation vector $q_t$ is computed as following:

$$q_t={\beta }_t\otimes h_t+h_t$$

(12)

3.3.4. Output and Loss Function

The TCN model in decoder outputs ${\tilde{H}}_T\in {ℝ}^{M\times O}$, where M denotes the total number of nodes and O represents the number of hidden neurons of the decoder TCN, and finally outputs the predicted adjacency matrix through the fully connected layer, as shown as follows:

$$\begin{array}{c}{\widehat{A}}_T=W_7{\tilde{H}}_T+b_3\end{array}$$

(13)

where $W_7$, $b_3$ denote the learnable weight matrix and bias vector, respectively.

The loss function consists of the prediction matrix ${\widehat{A}}_T\in {ℝ}^{M\times M}$ and the first-order adjacency matrix $A_T\in {ℝ}^{M\times M}$ of the real network topology at the next moment, and the loss function is defined as:

$$\begin{array}{c}L=L_{mse}+\lambda P_{nn}\end{array}$$

(14)

where $L_{mse}$ is the mean square error loss function, $P_{nn}={‖p‖}_1^2$ is the model parameter penalty term to prevent the model parameters from being too large or too small during the training process, and $\lambda$ is the learnable parameter. The mean square error $L_{mse}$ is minimized by optimizing the prediction model parameters $\theta$. $L_{mse}$ is calculated as shown as following:

$$\begin{array}{c}{L}_{mse}={\displaystyle \sum _t{‖e_t‖}_2^2}={\displaystyle \sum _t{‖A_t-{\widehat{A}}_t‖}_2^2}\end{array}$$

(15)

where $f\left({\tilde{A}}_1,\dots ,{\tilde{A}}_{T-1};\theta \right)$ is the predictive mapping function of the model, ${\widehat{A}}_T=f$ and $A_T={\widehat{A}}_T+e_T$, where $e_T$ is the prediction error.

Time-series prediction usually assumes that the errors are uncorrelated; however, due to the temporal correlation of the opportunistic network, the error $e_T$ is, in fact, auto-correlated, and this auto-correlated nature leads to an increase in the error of the final result, weakening the performance of the model, and the $\rho$th-order auto-correlated error is shown as following:

$$\begin{array}{c}{e}_T={\rho }_1{e}_{T-1}+\dots +{\rho }_p{e}_{T-p}+{ϵ}_T,\left|{\rho }_i\right|<1,\forall i\end{array}$$

(16)

where ${\rho }_1,\dots ,{\rho }_p$ is the auto-correlation coefficient, and ${ϵ}_T\in N\left(0,{\sigma }^2\right)$ is the non-correlated error, and the absolute value of ${\rho }_i$ is less than 1.

The first-order auto-correlation is the most significant term [6], because the correlation between $e_T$ and $e_{T-{\mathsf{\Delta }}_T}$ decreases when ${\mathsf{\Delta }}_T$ increases, so Formula (16) can be simplified to $e_T=\rho e_{T-1}+{ϵ}_T$, and the auto-correlation error $e_T=A_T-f\left({\tilde{A}}_1,...,{\tilde{A}}_{T-1};\theta \right)$ at the moment of T can be obtained from the mapping function in Formula (15), and in the same vein, the auto-correlation error $e_{T-1}=A_{T-1}-f\left({\tilde{A}}_0,...,{\tilde{A}}_{T-2};\theta \right)$ at the moment of T − 1, so Formula (16) can be updated to Formula (17):

$$\begin{array}{c}{A}_T-\rho {\tilde{A}}_{T-1}=f\left({\tilde{A}}_1,\dots ,{\tilde{A}}_{T-1};\theta \right)-\rho f\left({\tilde{A}}_0,\dots ,{\tilde{A}}_{T-2};\theta \right)+{ϵ}_T\end{array}$$

(17)

where ${ϵ}_T$ is the non-correlated error and $\rho$ is unknown, so instead of $\rho$, the right-hand side of Formula (17) can be approximated by transforming the right-hand side of Formula (17) using the value of the learnable parameter $\widehat{\rho }$ and using $\widehat{\rho }$ as the trainable parameter. Through this transformation, the transfer of errors in the time series is eliminated so that the current prediction results are no longer affected by past errors. The input form of the target sequence is $A_T-\widehat{\rho }{\tilde{A}}_{T-1}$ as shown in Formula (17), and in time-series prediction, the input sequence and the target sequence should have the same form, so Equation (17) can be updated to Formula (18):

$$\begin{array}{c}{A}_T-\widehat{\rho }{\tilde{A}}_{T-1}\approx f({\tilde{A}}_{T-1}-\widehat{\rho }{\tilde{A}}_{T-2},\dots ,{\tilde{A}}_1-\widehat{\rho }{\tilde{A}}_0;\theta )\end{array}$$

(18)

Therefore, the input sequence and the target sequence are of the form $A_T-\widehat{\rho }{\tilde{A}}_{T-1}$. The predicted output of the model is ${\widehat{A}}_T=A_T-\widehat{\rho }{\tilde{A}}_{T-1}$, and the auto-correlation error is adjusted to non-correlation error ${\widehat{A}}_T+\widehat{\rho }{\tilde{A}}_{T-1}$.

4. Experiments

The model proposed in this paper will be evaluated on three opportunistic network datasets and compared with time-series and auto-encoder methods.

4.1. Datasets

To verify the effectiveness of the proposed model, we conduct sufficient experiments on three opportunistic network datasets ITC, MIT, and Infocom06 [25].

ITC is known as Imote-Traces-Cambridge, it includes t traces of 50 students carrying small Bluetooth devices (iMotes) in office, conference, and urban environments over a 12-day period. The MIT dataset collected records of students carrying cell phones for communication on the MIT campus; the entire experiment collected nearly 9 months of communication records of persons carrying 97 cell phones. Infocom06 collected communication records of 97 participants involved in the IEEE Infocom conference over a period of 4 days. The information related to the three datasets is shown in

Table 1. Datasets used in this work.

Dataset	ITC	MIT	Infocom06
Data collection devices	iMote	Mobile phone	iMote
Number of nodes	50	97	97
Duration (day)	12	246	3
Type of communication	Bluetooth	Bluetooth	Bluetooth

.

4.2. Evaluate Metrics

The link prediction task can be regarded as a binary classification task. Therefore, in order to comprehensively evaluate the performance of the method, AUC, Precision, and F1-score are used to evaluate the performance of the algorithm. Three metrics are specifically defined below.

(1)

Area Under the Curve (AUC) [26]: AUC is a widely used metric in binary classification that represents the area under the receiver operating characteristic (ROC) curve. The ROC curve is a plot of the true positive rate against the false positive rate at various threshold settings. AUC provides an overall measure of the model’s ability to discriminate between positive and negative classes.

(2)

Precision: Precision is defined as the proportion of correctly predicted links in a prediction list of all samples. It is calculated as the ratio of true positive predictions to the sum of true positive and false positive predictions.

(3)

F1-score: The single Precision is difficult to effectively evaluate, so a comprehensive evaluation metric F1-score is used as a supplement, which simultaneously considers the precision and recall of the classification model. Where recall is defined as the proportion of correctly predicted links in the test set.

4.3. Results and Analysis

The experiments mainly consist of four parts: the decision on whether to consider link frequency in node attribute vector, the determination of slice size, ablation studies, and comparison of predictive performance with baseline models. The ablation studies include the validation of spatial-temporal attention and the validation of autocorrelated error adjustment. Experimentally, the dataset is sliced into a training set and test set in 7:3 ratio.

4.3.1. Decision on Whether to Consider Link Frequency

To decide whether to consider link frequency in attribute feature representation, this paper calculated the average duration of all the node pairs on three datasets. As shown in Figure 7, the average link duration in the ITC and MIT datasets is obviously higher than the one in the Infocom06 dataset.

Due to the presence of excessively long links (greater than one hour) and excessively low link frequency between a node pair (less than three links) in the datasets, which do not align with the characteristics of mobile device communication, we discard these abnormal communication data. Furthermore, for further analysis at different time slice sizes [27], the study calculated the number of links with an average link duration greater than the slice size. The proportion of this number to the overall links (remove outliers) is shown in

Table 2. Proportion of node pairs with the average link duration longer than the slice size.

Slicing Size (s)	ITC	MIT	Infocom06
240	86.67%	70.67%	10.77%
300	85.28%	64.85%	7.33%
360	84.24%	58.64%	5.14%
420	82.50%	52.07%	3.77%
480	82.03%	46.14%	2.78%
540	81.80%	40.89%	2.10%
600	72.76%	36.40%	1.68%

.

As can be seen from Table 2, the average link duration percentage greater than the duration of 600 s reaches 72.76% and 36.40% on the datasets ITC and MIT, respectively. Thus, only the link duration is considered in these two datasets; we can construct the node attribute vector through Formula (2). On the Infocom06 dataset, the average link duration greater than duration 240 s accounts for 10.77%; therefore, the attribute vector should take into account the link frequency, which can be constructed by Formula (3).

4.3.2. Determination of Slicing Size

The purpose of this experiment is to determine the optimal slice size of the STA-TCN prediction method on different datasets. To determine the effect of opportunistic network slice size on the performance of the STA-TCN prediction method, the prediction method is evaluated using AUC, Precision, and F1-score. The prediction results are greatly affected by the slice size, so it is necessary to select the appropriate slice size. If the slice size is too short, the network evolution is too slow, and the input data are prone to redundancy; if the slice size is too large, some of the evolutionary features of the network may be lost. Setting slice sizes from 4 min to 10 min with one-minute intervals, the experimental results are shown in Figure 8:

As shown in Figure 8, on ITC and MIT datasets, evaluation metrics decrease when the slice size increases. On Infocom06 dataset, evaluation metrics Precision and AUC are the maximum at T = 240 s, while F1-score reaches a maximum at T = 360 s. Therefore, the slice size on ITC, MIT, and infocom06 is set to 240 s.

4.3.3. Validation of Spatial-Temporal Attention

To validate the effectiveness of the spatial-temporal attention module, the following experiment is designed. On three datasets, our method STA-TCN is compared with the STA-TCN-NOATT method, which does not consider spatial-temporal attention. The opportunistic network slice size is set to 240 s in the ITC, MIT, and Infocom06 datasets, and other conditions remain the same. The prediction results are shown in the Figure 9:

In Figure 9, the y-axis denotes the values of evaluation metrics. From the results shown in Figure 9, on the ITC dataset, STA-TCN and STA-TCN-NOATT exhibit the same Precision, but STA-TCN achieves greater AUC and F1-score compared to STA-TCN-NOATT, indicating that spatial-temporal attention has a certain effect on performance improvement in the ITC dataset. Similarly, on the MIT dataset, STA-TCN and STA-TCN-NOATT show the same Precision, yet STA-TCN achieves greater AUC and F1-score values than STA-TCN-NOATT, demonstrating that spatial-temporal attention has a stronger impact on performance improvement in the MIT dataset. At last, on the Infocom06 dataset, STA-TCN achieves greater Precision, AUC, and F1-score values than STA-TCN-NOATT, indicating that spatial-temporal attention can effectively enhance prediction performance on the Infocom06 dataset. Hence, considering spatial-temporal attention in the prediction method significantly improves the model performance.

4.3.4. Validation of Non-Autocorrelated Errors

In the loss function construction, the auto-correlation error is adjusted to a non-correlation error, considering that the internal and external environments will generate an auto-correlation error on the prediction method, and this auto-correlation error will have an impact on the prediction results. In order to verify the effectiveness of the method of adjusting the auto-correlation error, the following experiments are designed to compare the STA-TCN method with the STA-TCN-NOE method without adjusting the auto-correlation error. In the ITC, MIT, and Infocom06 datasets, the opportunistic network slice size is set to 240 s, and other conditions are the same, and the experimental results are shown in Figure 10:

From the results in Figure 10, it is concluded that STA-TCN works better than STA-TCN-NOE on ITC and MIT datasets, and the enhancement of experimental effect is more obvious on MIT data. Although the evaluation index F1-score of STA-TCN-NOE in infocom06 is better than the one of STA-TCN, its Precision and AUC are worse than the one of STA-TCN. Therefore, adjusting the model autocorrelation error can improve the performance of the model.

4.3.5. Comparison of Link Prediction Performance

To ensure the reliability of the experimental results, we conducted 100 rounds of training and testing for each method on the three datasets. The comparison was based on the average values as shown in

Table 3. Comparing our methods with other methods.

Methods	Dataset	ITC			MIT			Infocom06
	Metric	Precision	AUC	F1	Precision	AUC	F1	Precision	AUC	F1
TCN		0.9123	0.9421	0.8684	0.9766	0.9755	0.9497	0.8165	0.8671	0.6121
LSTM		0.9333	0.9625	0.885	0.9824	0.955	0.9108	0.7058	0.8953	0.6283
DynRNN [15]		0.8312	0.9393	0.7869	0.927	0.9259	0.774	0.5057	0.829	0.4549
DynAERNN [15]		0.9293	0.9582	0.8828	0.8858	0.8795	0.7046	0.5903	0.8574	0.5262
E-LSTM-D [18]		0.9384	0.9592	0.8894	0.9494	0.9381	0.8483	0.7498	0.8927	0.6232
AGCRN [21]		0.7702	0.9535	0.6454	0.9502	0.9235	0.7493	0.4769	0.8629	0.3903
STA-TCN		0.8871	0.9644	0.8744	0.9506	0.9789	0.9462	0.7615	0.9105	0.6346

In Table 3, the bold values indicate that the corresponding method performs the best on the current dataset, while the underlined numbers represent the second-best performance of the corresponding method on the current dataset.

:

The compared methods are composed of time series models as well as their variants. As shown in Table 3, compared with the best baseline methods on the AUC metric, STA-TCN obtained a score improvement of 0.2–2% on the three datasets. It achieved an intermediate level of performance in terms of Precision on the three datasets. Additionally, it performed at above-average, suboptimal, and optimal levels on the F1 metric. Temporal models, such as TCN, LSTM, DynRNN, DynAERNN, and the temporal model E-LSTM-D, which combines autoencoders, only learn the evolution patterns of links in the opportunistic network without considering the interactions between nodes and ignoring the spatial dependencies between nodes, leading to information loss. The method AGCRN which combines GNN with RNN, uses an adaptive adjacency matrix to construct initial features, but it is unable to learn the rich latent features of nodes. Our method fully extracts the original features of the opportunity network through the state matrix and integrates the temporal and spatial influences with the help of the spatial-temporal attention mechanism, so STA-TCN achieves better results than other methods.

STA-TCN achieved the highest AUC and F1 values and the second highest accuracy on the Infocom06 dataset, as well as the highest AUC and the second highest F1 value on the MIT dataset. It can be seen that STA-TCN performs best on the Infocom06 dataset and second best on the MIT dataset. While on the ITC dataset, it does not perform as well as a single temporal model in Precision and F1 metrics, which is due to the fact that the size of the ITC dataset is smaller than the other two and the addition of the state matrix and the spatial-temporal attention may add some redundant information.

Overall, our method achieved the highest value in the AUC evaluation metric on all three datasets. AUC measures the model’s ability to distinguish between positive and negative samples, and it is a good indicator of overall model performance; a higher AUC value indicates better model performance. In addition, our method also performs generally at an above-average level on other evaluation metrics. Thus, it can be concluded that our method achieves an improvement compared to the baseline methods.

5. Conclusions

In this paper, we proposed a link prediction method called STA-TCN, which is based on temporal convolutional network and attention mechanism. A state matrix is taken to learn the potential node attribute information in the opportunistic network. Spatial-temporal attention mechanism is employed to capture the spatial-temporal dependencies in the opportunistic network. Auto-correlation error is transformed into non-correlated error during the prediction process to enhance the model’s predictive capability. On three real opportunistic network datasets, STA-TCN demonstrates superior AUC as well as good precision and recall values. Therefore, the proposed model can effectively improve the performance of link prediction in opportunistic networks. However, the proposed model does not take into account the heterogeneity of networks in the real world. Future research will extend the study into heterogeneous networks for more application scenarios.