Simultaneously Captures Node-Level and Sequence-Level Features in Parallel for Cascade Prediction

Abstract

Predicting information diffusion in social networks is a fundamental problem in many applications, and one of the primary challenges is to predict the future popularity of information in social networks. However, most existing models fail to simultaneously capture the accurate micro-level user node features, meso-level linear spread features, and predict the macro-level popularity during the information propagation process, which may result in unsatisfactory prediction performance. To address this issue, we propose a new cascade prediction framework CasNS: Node-level and Sequence-level Features for Cascade Prediction. CasNS utilizes node-level features by employing a self-attention mechanism to capture the hidden features of the target node with respect to other nodes. Additionally, it leverages multiple one-dimensional convolutional layers with the dynamic routing algorithm to obtain sequence-level features across different dimensions. Through experiments on a large number of real-world datasets, our model demonstrates superior performance compared with other state-of-the-art methods, thereby validating the feasibility of our approach.

1. Introduction

The ability to accurately predict the propagation range of information is crucially important for monitoring social networks. However, accurate predictions are typically a challenging task, as they require modeling complex systems with many interdependent variables. Deep learning technologies [1,2,3,4,5] have made significant progress in improving predictive accuracy in recent years, although there is still room for improvement.

One promising way to improve predictive accuracy is cascade prediction, which involves decomposing a complex prediction task into a series of smaller, more tractable subtasks. In this approach, each subtask predicts a particular aspect of the final outcome and the outputs of the subtasks are combined to form the final prediction.

Cascade prediction has several advantages over traditional prediction methods. First, it allows for more flexibility because each subtask can be customized for specific features of that aspect of the system. Second, it can improve predictive accuracy by reducing errors in individual subtasks. Finally, cascade prediction can achieve faster and more efficient predictions by decomposing the problem into smaller, more tractable components.

Nowadays, a growing body of research has investigated cascade prediction from different perspectives. A primary approach in the literature involves modeling information diffusion using random point processes, where events are assumed to occur stochastically in continuous space or time. Representative studies [6,7,8,9,10] leverage spatial or temporal point process formulations to estimate future cascade growth by modeling event distributions. Another significant direction involves mechanism-driven models that seek to explain the formation of propagation patterns. For example, recent work by Zhu et al. [11] has applied reaction-diffusion systems to analyze spatio-temporal dynamics, ranging from epidemic modeling on higher-order networks to the propagation of behaviors through a structured three-layer network model (information-awareness-behavior) [12]. These models excel at providing mechanistic explanations. Another set of methods relies on feature engineering, in which handcrafted features are extracted from cascades, users, or network structures and subsequently fed into predictive models [13,14,15,16,17]. While these approaches can achieve reasonable performance, they often require extensive domain knowledge and auxiliary information that may be unavailable in practice due to privacy or data-access constraints. More recently, deep learning-based methods have achieved remarkable success with the rapid development of GPU computing. Models based on recurrent neural networks (RNNs) [1,2,3] and graph neural networks (GNNs) [4] have been proposed to capture temporal dependencies and structural relationships in diffusion processes, significantly outperforming traditional feature-based methods.

Despite these advances, three key limitations remain. First, many methods have strict requirements for the datasets to a greater or lesser extent, including the introduction of time parameters as sequence weight parameters, the observed cascade size within a period of time, etc. Second, most models cannot accurately capture the embedding relationships between nodes and other nodes, both in the local and global graph. Third, Most models cannot obtain the multi-dimensional embedding of the entire sequence.

To solve the above problems, this paper proposes a novel cascade prediction framework, Node-level and Sequence-level Features for Cascade Prediction (CasNS), as shown in Figure 1. Our method involves training a series of neural networks, each network responsible for predicting a particular aspect of the final result, and then combining the outputs of these networks into the final prediction. Our method has the following three advantages and innovations over traditional prediction methods:

• Loose requirements for data: Traditional cascade prediction methods often require a lot of data sets to achieve decent prediction results, and have very strict requirements for datasets. For example, CasCN [4] requires that the training and prediction data sets must be cascade graphs with no more than 100 cascades within the observation time. DeepHawkes [3] needs to introduce time parameters in order to greatly improve the performance of the model compared with DeepCas [2] and other traditional cascade prediction models. However, the model we proposed in this paper only needs a small amount of training data and the datasets do not need time parameters for training a good performance model.
• Accurately capturing local and global features of nodes: Traditional cascade prediction methods simply use recurrent neural networks or graph convolutional neural networks, which can only capture the embedding relationship between nodes and their adjacent nodes, and cannot directly capture the embedding relationship between nodes and all nodes in the sequence. The proposed cascade prediction framework introduces a Transformer, which can capture the embedding relationship between each node in the sequence and all other nodes in the sequence.
• Accurately capturing the multi-dimensional features of nodes: Traditional cascade prediction frameworks, such as DeepCas [2], and Deephawkes [3], obtain the embedding of the entire sequence by summing weighted sequence nodes, and the weight parameters are difficult to learn. The proposed cascade prediction framework introduces a dynamic routing algorithm to directly capture the multi-dimensional embedding of the entire sequence, and it makes the obtained sequence embedding more usable.

The paper will be organized as follows. In Section 2, we will introduce and analyze existing research. In Section 3, we will first present the relevant definitions and then gradually elaborate on our proposed model, CasNS. Section 4 will describe the experiments we conducted, including performance comparisons and analysis, as well as experiments on model interpretability. The final section will provide a summary and outline of our future work.

2. Related Work

In this section, we will review the achievements that researchers have made in cascade prediction, transformer, and capsule neural networks.

2.1. Cascade Prediction

Existing works on predicting information cascade propagation can be divided into three categories of methods: methods based on random point process, methods based on feature engineering, and methods based on deep learning.

2.1.1. Methods Based on Random Point Process

The concept behind these methods is to consider the dissemination of information as the arrival of user forwarding and sharing events. Shen et al. [6] propose a generative probabilistic framework that utilizes reinforced Poisson processes to model the process of individual item popularity, this framework provides an intuitive modeling approach to capture the popularity dynamics of individual items. Lu et al. [7] propose a generative framework that contains a hidden user interest layer, aiming to capture the collective behavior of users in the process of information propagation. Sreenivasan et al. [8] build a model of information cascades on feed-based networks, taking into account the finite attention span of users, message generation rates, and message forwarding rates. They find that the extent of user attention affects the probability that the cascade becomes viral. Li et al. [9] combine visual reasoning research with an enhanced point process-based model to strengthen the connections between users and prediction models, aiming to achieve better prediction results. Kong et al. [10], concurrently, another significant research direction employs mechanism-driven mathematical models. Notably, the work of Zhu et al. exemplifies this approach. They employ reaction-diffusion equations to analyze pattern formation, whether modeling epidemics on higher-order networks [11] or capturing the complex dynamics of behavioral change through a sophisticated three-layer model coupled with the Microscopic Markov Chain Approach (MMCA) [12]. In contrast, our data-driven CasNS framework is designed not to explicitly model such multi-layered causal mechanisms, but to predict the final cascade size by implicitly learning complex propagation dynamics from observational data.

2.1.2. Methods Based on Feature Engineering

Feature engineering in these methods typically involves artificially designing and extracting features engineered for a particular dataset, such as the features of users [13,14], the content features of propagated information [15], the graph structural features of information propagation [16] and temporal features [17]. The core of these methods lies in whether related graph structural features can be selected. Alweshah et al. [18] adopt the monarch butterfly optimization (MBO) algorithm and implemented it with a wrapper feature selection method using the k-nearest neighbor (KNN) classifier. For example, Zhao et al. [19] proposed the GAUG framework, which learns to augment the graph by adding or removing edges via a neural predictor to improve GNN-based node classification. Xiao et al. [20] study the influence of information release time on information popularity, establishing a time-sensitive prediction model to predict post popularity values when posted at different times. Carta et al. [21] propose an original method based on gradient boosting and feature engineering to tackle the problem of predicting future post popularity on Instagram.

2.1.3. Methods Based on Deep Learning

Prediction methods based on deep learning avoid the high cost and instability of artificially designing features and are currently the primary research methods for researchers [22,23,24,25,26,27]. Typically, researchers transform the graphs of information propagation into sequences, using recurrent neural networks (RNNs) [1] to model the sequences. DeepCas [2] is the first cascade prediction model based on deep learning, effectively combining GRU and MLP to obtain high-quality cascade graph predictions and greatly improving model performance compared with traditional methods. DeepHawkes [3] improves upon DeepCas by incorporating self-exciting processes and using time parameters as weights to obtain cascade graph labels, further enhancing performance. CasCN [4] directly uses graph neural networks to obtain predicted cascade graph labels, surpassing models that only use recurrent neural networks. GTGCN [5] combines graph neural networks and recurrent neural networks to capture higher-quality graph embeddings. While these methods improve prediction accuracy, user influence and community redundancy are often not studied in depth.

2.2. Multi-Head Self-Attention Mechanism

By adopting a multi-head self-attention mechanism to process time series, we can directly capture the embedding relationships between any two nodes in the sequence, not just adjacent nodes. Vaswani et al. [28] first introduce the transformer, and it deals with the machine translation task. It proposes using multi-head attention to capture interdependencies between sequences and using positional encoding to obtain positional information. Dai et al. [29] handle sequences longer than the original Transformer’s sequence length and achieved the latest results on tasks such as language modeling and sequence prediction. Wang et al. [30] apply the transformer to time series prediction and increase the temporal sparsity mechanism to improve efficiency. Zuo et al. [31] replace the positional encoding with temporal encoding in the Transformer encoder part for predicting the timing and probability of events over time. Zhou et al. [32] introduce the sequence down-sampling mechanism in Transformer for processing long-time sequences and named it “Informer”. These achievements all indicate that Transformer has outstanding performance in handling time series tasks. Therefore, to accurately capture the influence of each node in the sequence, we introduce a Transformer into our model.

2.3. Dynamic Routing Mechanism

Hinton et al. [33] first propose capsule networks and put forward the basic idea of capsule networks and the dynamic routing mechanism. Jayasekara et al. [34] extend the original dynamic routing mechanism to the time domain and proposed Time-Series Dynamic Routing for time-series classification. Elhalwagy et al. [35] combine capsule networks and autoencoders and proposed Capsule Autoencoders for anomaly detection in time series data. Wu et al. [36] apply capsule neural networks to recommender systems, where they use capsule networks to directly capture the embedding of the entire event sequence. Capsule Neural Networks have strong abilities to capture overall embedding. Therefore, to capture the overall embedding of the sequence, we introduce Capsule Neural Networks into our model.

3. Method

In this section, we will first provide the relevant definitions and define the problems we aim to address. Subsequently, we will provide a detailed description of our model framework.

3.1. Relevant Definition

The basic definitions are as follows:

• Cascade Graph: Suppose we have a total of M message propagation paths, denoted as $M=m^i(1\le i\le M)$. For each message $m^i$, we use a cascade graph $g_C^T=\left(E_C^T{V}_C^T\right)$ to record the diffusion process of message $m^i$, where $V_C^T$ is a subset of nodes in V, and $E_C^T=E\cup \left(V_C^T\times V_C^T\right)$ is the set of edges, as shown in Figure 2. A node $u$ in $V_C^T$ represents a cascade participant, such as a citation relationship in a paper citation network or a user involved in message reposting in a social network. An edge $(u,v)\in V_C^T$ represents an interaction between nodes u and v.
• Cascade Prediction: Given the information propagation processes and paths within the observation time window [0, T), our objective is to predict the incremental popularity $\Delta R_T^i$ between the observed popularity $R_T^i$ and the final popularity $R_{\infty }^i$ of each cascade graph $g_C^T$. This represents the popularity of the information beyond the observation time. To avoid the inherent correlation between the final popularity and the observed popularity, we choose to predict the incremental popularity,

3.2. Proposed Model

Our cascade prediction model takes the cascade graph $g_C^T$ within the observed time as input and predicts the cascade size $\Delta R_T^i$ outside the observed time. Our cascade prediction framework consists of the following four parts:

• Embedding Initialization: Traverse the cascade graphs observed within the observed time, convert them into sequences, and transform the nodes in the sequences into vectors of predefined dimensions.
• Learning micro-level features between nodes: By leveraging the inter-node self-attention mechanism, we capture the hidden features of the target node and all other nodes in the sequence at a micro-level. This allows us to update the embedding of the target node.
• Learning meso-level features in a sequence: By utilizing multiple parallel one-dimensional convolutional layers, we obtain sequence-wide features from different dimensions at a meso-level. We then use the dynamic routing algorithm to aggregate weighted sums of sequence features from different dimensions, thereby updating the sequence features.
• Path Encoding and Sum Pooling: Use GRU to extract the embedding relationship features between nodes in the sequence and their neighboring nodes, and sum them to obtain the features of the entire cascading graph.
• Prediction Module: Utilize dense fully connected layers to output the predicted cascade size $\Delta R_T^i$.

The above module decomposition reflects the different levels of information involved in cascade prediction. Specifically, micro-level inter-node interactions, meso-level sequence patterns, and path-level temporal dependencies exhibit distinct characteristics and are, therefore, modeled by different mechanisms. Rather than relying on a single uniform encoder, CasNS adopts specialized components(such as multi-head attention, GRU and capsule networks) at each level to capture complementary aspects of diffusion dynamics, which together form a coherent and expressive representation for cascade size prediction.

3.3. Embedding Initialization

Given a cascade graph $g_C^T$, our model first converts it into an easily processable sequential structure by traversing the entire graph using Breadth-First Search. Thus, the representation of the original $g_C^T$ is a set of node sequences. For example, traversing the cascade graph $g_C^T$ requires N sequences, and the longest sequence contains M nodes. We can represent the cascade graph $g_C^T$ as $G\in {\mathbb{R}}^{N\times M}$. Next, we use vectorized representations for each node ID. For each user’s ID, we represent it as $e_u^{\left(0\right)}\in {\mathbb{R}}^D(e_i^{\left(0\right)}\in {\mathbb{R}}^D)$, where D is the embedding dimension for each user. In this way, we can represent a sequence as $Y\in {\mathbb{R}}^{M\times D}$. Thus, we can represent a cascade graph as $E\in {\mathbb{R}}^{N\times M\times D}$.

3.4. Learning Meso-Level Features in a Sequence

3.4.1. Positional Encoder

The key component of our proposed model is the self-attention module. Unlike RNN, the attention mechanism does not discard periodic structures. However, our model still needs to consider the positional relationships between nodes. Therefore, we incorporate position encoding on top of the initial embeddings, and the calculation is as follows:

$$P{E}_{(pos,2i)}=sin({\displaystyle \frac{pos}{{10000}^{\frac{2i}{d_{model}}}}}),$$

(1)

$$P{E}_{(pos,2i+1)}=cos({\displaystyle \frac{pos}{{10000}^{\frac{2i}{d_{model}}}}}),$$

(2)

where $pos$ refers to the position of a node in the sequence, $d_{model}$ represents the dimension of the node vectors, in the case of the cascade graph $g_C^T$, we have $d_{model}=D$, i denotes the position of the node vector. Specifically, for each node u in the sequence, we deterministically calculate its positional encoding $z\left(\mathrm{pos}\right)\in {\mathbb{R}}^D$. Then, $U\in {\mathbb{R}}^{M\times D}$ represents the positional encoding of the sequence.

3.4.2. Multi-Head Attention

After the initial encoding and position encoding, we define $X=(Y+U)\in {\mathbb{R}}^{M\times D}$ to pass through the self-attention module. Specifically, we compute the attention output S by

$$S=Softmax\left({\displaystyle \frac{Q{K}^T}{\sqrt{M_K}}}\right)V,$$

(3)

$$Q=X{W}^Q,$$

(4)

$$K=X{W}^K,$$

(5)

$$V=X{W}^V,$$

(6)

here, Q, K and V are the query, key, and value matrices obtained by applying different linear transformations on the sum of the initialized embeddings and positional encodings of the sequence, denoted as X. The matrices $W^Q\in {\mathbb{R}}^{D\times M_Q},W^K\in {\mathbb{R}}^{D\times M_K},W^V\in {\mathbb{R}}^{D\times M_V}$ represent the weights for the respective linear transformations applied to X. In our experiments, the inclusion of multi-head self-attention allows for a more detailed capture of the embedding relationships between nodes. To achieve this, we use different sets of weights $S_1,S_2,\dots ,S_H$ to compute different self-attention outputs, denoted as ${\left\{W_h^Q,W_h^K,W_h^V\right\}}_{h=1}^H$. Finally, our output for the event sequence is given by the following:

$$S=\left[S_1,S_2,\dots ,S_H\right]W^O,$$

(7)

where $W^O$ is an aggregation matrix.

The j-th column of the attention weights $Softmax\left({\displaystyle \frac{Q{K}^T}{\sqrt{M_K}}}\right)$ represents the dependency of user $u_j$ on other nodes $u_i$. In contrast, models based on RNN encode user node information sequentially using the hidden representation of users. This means that the state of user $u_j$ depends on the state of user $u_{j-1}$, which in turn depends on the state of user $u_{j-2}$, and so on. If any of these encodings are weak, meaning that the RNN fails to learn sufficient information related to user $u_k$, then the hidden representation of any user $u_j$ with $j\ge k$ will be poorer.

3.4.3. Position-Wise Feed-Forward Networks

Afterward, the attention output S is processed through a position-wise feed-forward neural network to generate the hidden representation $h\left(u\right)$ of the input user sequence:

$$P=ReLU\left(S{W}_1^{FC}+b_1\right)W_2^{FC}+b_2,$$

(8)

$$p\left(u_j\right)=H\left(j,:\right),$$

(9)

where $W_1^{\mathrm{FC}}\in {\mathbb{R}}^{D\times M_H}$, $W_2^{\mathrm{FC}}\in {\mathbb{R}}^{M_H\times D}$, $b_1\in {\mathbb{R}}^{M_H}$, and $b_2\in {\mathbb{R}}^D$ are the parameters of the neural network. Notably, $W_2^{\mathrm{FC}}$ has the same number of columns. The resulting matrix $P\in {\mathbb{R}}^{M\times D}$ contains the hidden representations of all user nodes in the input sequence, where each row corresponds to a user.

3.4.4. Compression Encoder

To enhance the expressive power and performance of the Multi-head Attention model and enable its training parameters to better fit nonlinear and complex functional relationships, we introduce a non-linear squashing function at the end:

$$S\left(h\right)={\displaystyle \frac{{\Vert h\Vert }^2}{1+{\Vert h\Vert }^2}}{\displaystyle \frac{h}{\Vert h\Vert }}.$$

(10)

This function normalizes the hidden representation h, which helps to stabilize the learning process in subsequent modules.

3.4.5. Gated Multi-Modal Units

In the above, we first initialize the hidden representation of the sequence nodes and then utilize self-attention mechanisms between nodes to obtain hidden embeddings of the nodes with respect to other nodes in the sequence. However, a question arises regarding how to aggregate these updated hidden representations.

We use a gate mechanism similar to LSTM, as shown in Figure 3. It is a multimodal learning model based on gate-controlled neural networks. The Gate-Multi-modal Unit (GMU) model is designed to serve as an internal unit in neural network architectures with the purpose of finding intermediate representations based on combinations of data from different modalities. The GMU learns to use multiplication gates to determine how each modality influences the unit’s activation, and the computation is as follows:

$$l_y=tanh\left(W_y·Y\right),$$

(11)

$$l_p=tanh\left(W_p·P\right),$$

(12)

$$z=\sigma \left(W_Z·\left[l_y,l_p\right]\right),$$

(13)

$$L=z·Y+(1-z)·P,$$

(14)

for any sequence l, its initialized sequence representation is denoted as $Y\in {\mathbb{R}}^{M\times D}$. After updating the node features using self-attention mechanisms between nodes, the representation becomes $H\in {\mathbb{R}}^{M\times D}$. The training parameters for the GMU are denoted as $W_y,W_h,W_z\in \mathbb{R}$. For any sequence in the cascade graph, its sequence representation through the GMU is denoted as L, and the hidden representation of node $u_j$ in the sequence is as follows:

$$l\left(u_j\right)=L\left(j,:\right).$$

(15)

3.5. Learning Micro-Level Features Between Nodes

To capture the entire sequence features, we propose a sequence-specific capsule module to overcome the limitations of previous work, which is insensitive to order (i.e., direction and position). The module consists of a horizontal convolution and a dynamic routing process. The input to the module is the initialized embedding of the sequence nodes $E=[e_1,e_2,\dots ,e_n](e_i\in {\mathbb{R}}^D)$. Borrowing the idea of CNN on knowledge graphs [37], we regard the specific matrix E of sequence l as an “image” for horizontal convolution.

As shown in Figure 1, our module contains two 1D convolutional layers and a routing algorithm. The first 1D convolutional layer Conv1 uses ReLU as the activation function. The purpose of this layer is to transform individual node features in the sequence into local sequence features, which are then passed as input to the second 1D convolutional layer.

The second layer consists of multiple one-dimensional convolutional layers that form the primary capsules, with each convolutional layer being identical. Each convolutional layer is used to capture features of specific dimensions in the sequence. The final layer (DigitCaps) aims to fuse the obtained features from different dimensions of the sequence through a routing algorithm.

We encapsulate all elements of the same dimension into a sequence-specific capsule $c_l^{\left(d\right)}$ and obtain D capsules accordingly [$c_l^{\left(d\right)}|d\in 1,\dots ,D$]. Each capsule $c_l^{\left(d\right)}$ undergoes a linear transformation via the corresponding weight matrix $W_c$ to produce a “prediction vector” $\widehat{c_l^{\left(d\right)}}$. These vectors $\widehat{c_l^{\left(d\right)}}(d\in 1,\dots ,D)$ are added together to generate a capsule $s_l$. This capsule performs a nonlinear squashing function to produce vector output ${z^{\prime }}_l$. Formally, we have the following:

$$z_l^{\prime }=squash\left(s_l\right),$$

(16)

$$s_l=\sum _{d=1}^D{\beta }_d\widehat{c_l^{\left(d\right)}},$$

(17)

$$\widehat{c_l^{\left(d\right)}}=W_c^{\left(d\right)}{c}_l^{\left(d\right)},$$

(18)

where $squash\left(s_l\right)={\displaystyle \frac{\Vert s_l{\Vert }^2}{1+{\Vert s_l\Vert }^2}}{\displaystyle \frac{s_l}{\Vert s_l\Vert }}$, the squashing function here is a standard component of the capsule network’s dynamic routing algorithm. While its form is identical to the one used in Section 3.4.4, its core role here is to enable the dynamic routing mechanism. ${\beta }_d$ is the coupling coefficient representing the importance of capsule $c_l^{\left(d\right)}$, which is determined by the dynamic routing process as shown in Algorithm 1.

Algorithm 1: Dynamic Routing Process

We now obtain the hidden state $z_l^{\prime }\in {\mathbb{R}}^H$ of sequence l. And the embedding representation of the information propagation paths formed by N sequences is denoted as $Z\in {\mathbb{R}}^{N\times H}$.

3.6. Path Encoding and Sum Pooling

After obtaining the hidden representations of the target node and other nodes in the sequence using the self-attention mechanism, we need to obtain the representation of the entire sequence at a micro-level. Here, we perform path encoding on the hidden states of nodes in the sequence and use GRU for aggregation to obtain the hidden state of the entire sequence.

To capture the evolving relationships in the information dissemination process, we employed Gated Recurrent Units (GRU), which is a specific type of Recurrent Neural Network (RNN) known for its strong performance in sequence modeling. When using GRU for modeling sequences from left to right, the representation of subsequent sequence nodes will incorporate information from previous nodes. The gate mechanism in GRU determines the proportion of new information and historical information embedded in the sequence nodes, simulating the flow of information during the diffusion process. Specifically, for the i-th node in the sequence, assuming the input node embedding $l_i\in R^D$ and the previous hidden state $h_i\in {\mathbb{R}}^D$ as inputs, the GRU computes the updated hidden state $h_i=GRU(s_i,h_{i-1})$, where $h_i\in {\mathbb{R}}^D$:

$$updategate: u_i=\sigma (W_u{s}_i+U_u{h}_{i-1}+b_u),$$

(19)

$$resetgate: r_i=\sigma (W_r{s}_i+U_r{h}_{i-1}+b_r),$$

(20)

$$candidatestate: \widetilde{h_i}=tanh(W_h{s}_i+U_h(r_i\odot h_{i-1})+b_h),$$

(21)

$$hiddenstate: h_i=u_i\odot \widetilde{h_i}+(1-u_i)\odot h_{i-1},$$

(22)

in this context, the symbol ⊙ denotes element-wise multiplication, where $W_u,W_r,W_h,U_u,U_r,$$U_h\in {\mathbb{R}}^{D\times D}$ represent the matrix parameters in the GRU, and $b_u,b_r,b_h\in {\mathbb{R}}^D$ are the bias parameters in the GRU.

A unidirectional GRU models the sequence in a forward time order, where subsequent nodes contain progressively richer features. However, the forward nodes have little knowledge of which nodes they will influence, which does not align with the flow of information in the diffusion process. Therefore, we introduce a bidirectional GRU to address this limitation, allowing the preceding nodes to know which subsequent nodes they will propagate information to. The representation of the i-th node in the k-th sequence:$\stackrel{⟷}{h_{s_i}^k}\in {\mathbb{R}}^{2D}$ is computed using the following equations:

$$\stackrel{⟷}{h_{s_i}^k}=\stackrel{⟵}{h_{s_i}^k}\oplus \stackrel{⟶}{h_{s_i}^k},$$

(23)

$$\stackrel{⟵}{h_{s_i}^k}=GR{U}_f(s_i,\stackrel{⟵}{h_{s_{i+1}}^k}),$$

(24)

$$\stackrel{⟶}{h_{s_i}^k}=GR{U}_f(s_i,\stackrel{⟶}{h_{s_{i-1}}^k}),$$

(25)

where the symbol ⊕ represents the concatenation operation, $\stackrel{⟵}{h_{s_i}^k}$ and $\stackrel{⟶}{h_{s_i}^k}$ represent the forward and backward hidden vectors, respectively. The k-th sequence with length L can be denoted as $[\stackrel{⟷}{h_{s_1}^k},\stackrel{⟷}{h_{s_2}^k},\dots ,\stackrel{⟷}{h_{s_n}^k}]$. Then, we take the last node’s embedding $\stackrel{⟷}{h_{s_n}^k}$ to represent the hidden state of the whole sequence. For a cascade graph with N sequences, its hidden state is as follows:

$$R=\sum _{n=1}^N\stackrel{⟷}{h_{s_n}^k}.$$

(26)

3.7. Prediction Module

The last part of our model consists of a multi-layer perceptron (MLP) and an adder that takes the hidden state R and P of the cascade graph as input and outputs a final prediction:

$$\widehat{\Delta R_T^i}=MLP(R+Z),$$

(27)

and our goal is to minimize the loss function:

$$loss={\displaystyle \frac{1}{K}}\sum _{k=1}^K{(log(\Delta R_T^i)-log\left(\widehat{\Delta R_T^i}\right))}^2,$$

(28)

where $\widehat{\Delta R_T^i}$ is the predicted incremental popularity of cascade $g_C^T$ by our model, $\Delta R_T^i$ is the ground-truth incremental popularity, and K is the total number of cascades. We take the logarithm of the incremental popularity because the original squared loss is prone to being influenced by outliers, while the transformed objective is similar to MAPE (Mean Absolute Percentage Error) and easier to optimize.

3.8. Framework Operational Mechanism

To concretely illustrate the execution of the CasNS framework, this section provides a detailed description of the data flow when processing a single cascade; Figure 1 also depicts the corresponding data movements during runtime.

The process begins with an original cascade graph. Using breadth-first temporal sampling, we extract N node sequences, which are subsequently padded to a uniform length M. Next, the N sequences are mapped by a shared node-embedding layer into a tensor of shape $[N,M,D]$. This initial tensor serves as the input basis for two parallel feature-extraction branches in the model.

Each of the N sequences is fed simultaneously into both branches for parallel processing. The meso-level sequence feature branch employs a capsule network: the input sequence tensor is processed by convolutions followed by the capsule network’s dynamic routing procedure to capture global, holistic characteristics of the entire sequence. The micro-level node feature branch first adds positional encodings to the input tensor and then processes the result with a multi-head self-attention-based encoder to produce context-aware node representations. A Gated Multimodal Unit (GMU) fuses the original embeddings with these context-aware embeddings to yield a refined tensor, which is finally passed into a bidirectional GRU.

After all N sequences have been processed, the model aggregates the outputs from each branch. The two aggregated vectors are concatenated and supplied to the MLP layer to produce the model’s prediction.

4. Experiment

This section conducts experiments on the two benchmark datasets introduced and compares with state-of-the-art baseline models for cascade prediction to evaluate our model. Ablation experiments are also carried out to analyze the contributions of different modules in our model.

4.1. Dataset

We apply our proposed model to two different cascade prediction scenarios to evaluate its performance. The first scenario predicts retweet cascades on Sina Weibo, and the second scenario predicts tweet propagation cascades on Twitter.

Table 1. Descriptive statistics of two datasets.

Dataset	Sina Weibo	Twitter
Number of Cascades and Nodes in Different Observation Settings
Train (1 h/1 day)	831	9639
Val (1 h/1 day)	178	2066
Test (1 h/1 day)	178	2066
Nodes in $G_g$	56,065	271,792
Train (2 h/2 day)	908	12,739
Val (2 h/2 day)	194	2730
Test (2 h/2 day)	194	2730
Nodes in $G_g$	66,422	370,947
Train (3 h)	927
Val (3 h)	198
Test (3 h)	198
Nodes in $G_g$	70,516

lists the statistical information of the datasets.

Sina Weibo: This dataset collected and provided by Cao et al. [3] can be openly accessed on their website. This dataset includes Weibo messages with more than 10 retweets, which were posted on 1 June 2016, along with the retweets received within 24 h of the original post. In addition to user IDs, the retweet information includes the retweet path and the time of retweeting. We consider cascades that were posted between 8:00 and 18:00 and have less than 1000 retweets as candidates for generating the global network structure, denoted as G. To validate our model’s ability to achieve high performance with a small training set, we randomly select over 1000 cascades that meet the aforementioned criteria. The specific number of cascades depends on the observation time. Next, the dataset is divided into training, validation, and testing sets based on the chronological order of message publication. Specifically, the first 70% of the data are used as the training set, the middle 15% are used as the validation set, and the remaining 15% are used as the testing set. The observation time length, denoted as T, is set to 1, 2, or 3 h, respectively.

Twitter: This dataset, collected by Weng et al. [38] includes publicly available English-written tweets posted between 24 March and 25 April 2012. We consider topic hashtags and their adopters as independent information diffusion processes. The global graph of the Twitter dataset is constructed using various relationships, including mutual followings/followers between users, retweeted tweets, and mention interactions. The cascade graph is built based on these three relationships. To validate our model’s performance on large-scale data, we select all cascades that meet the requirements. The observation time length, denoted as T, is set to 1 or 2 days, respectively.

4.2. Baseline Methods

To validate that our model has strong predictive performance on real-world datasets, we also experimented with other advanced methods on the two datasets, but these other methods were not able to simultaneously capture the Node-level and Sequence-level features of the sequences.

Feature-Linear is the most widely used information diffusion prediction model. These models typically involve manual feature extraction from the data, followed by training and evaluation using machine learning models. For instance, Szabo and Huberman [39] used observed popularity, denoted as $P_j\left(t_o\right)$, to predict the predicted popularity, denoted as $\widehat{P_j\left(t_p\right)}$, of news articles and online videos. They utilized observed popularity and cumulative popularity as features. These features were then fed into linear regression and MLP models to predict the popularity.

DeepCas [2] is the first model that truly applies deep learning to cascade prediction. Its primary goal is to predict the future popularity of information by analyzing the paths of information propagation. Specifically, it first processes the cascade graphs into multiple sequences, then uses recurrent neural networks to obtain embeddings of the sequences, and finally obtains the embedding of the cascade graph by weighted summing of all sequences. However, its weighted summing is based only on mathematical assumptions rather than the actual structure of cascade graphs.

DeepHawkes [3]. A model is proposed based on DeepCas by adding Hawkes processes [40]. It replaces the weighted summing of sequences in DeepCas based on mathematical assumptions with weight summing based on cascade diffusion times, which is closer to reality and enhances the interpretability of the model.

CasCN [4] is a model similar in concept to DeepHawkes, as they are both built based on self-exciting processes. CasCN introduces graph convolutional neural networks (GCNs) on top of DeepHawkes to directly aggregate node features in the graph and obtain global features of the graph. However, due to the varying number of nodes in different information cascades, the constructed Laplacian matrix has different dimensions. This constraint limits CasCN to only use cascade graphs with observed node counts ranging from 10 to 1000 in their experiments. This requirement for dataset selection is quite strict, and our model aims to avoid such limitations.

GTGCN [5] is a graph-based time-aware information learning framework that builds upon an enhanced graph convolutional network (GCN). It combines GCN with a recurrent neural network (RNN) in a linear fashion and introduces time parameters to capture temporal information governing information propagation within snapshots, as well as inherent temporal dependencies between different snapshots.

CasFlow [41] is a cascade prediction framework that obtains high-quality node embeddings by learning the latent representations of cascade graph structural information and cascade diffusion time information. Additionally, CasFlow can capture the uncertainty in information propagation and realize hierarchical learning of information diffusion patterns by doing so.

CasHAN [42] builds upon DeepCas and incorporates node-level self-attention mechanisms and community detection to capture meso-scale features in information propagation, while removing redundant nodes during the cascade prediction process.

The selected baselines cover representative paradigms ranging from traditional feature-engineering methods to modern deep learning approaches. Feature-Linear represents classical feature-based methods and serves as a reference for highlighting the overall advantages of deep learning. DeepCas [2] and DeepHawkes [3] are pioneering RNN-based models that focus on sequential modelling of propagation paths, enabling a comparison with our model in terms of sequence-level feature capture. CasCN [4] operates directly on graph structures and provides a key baseline for node-level feature modeling. GTGCN [5] and CasFlow [41] adopt hybrid strategies that integrate graph topology with temporal dynamics, reflecting an important research direction. CasHAN [42] incorporates attention mechanisms and constitutes the most direct comparison to our node-level self-attention module. Overall, these baselines ensure both the relevance and diversity of the experimental comparison.

4.3. Experimental Setup

We will measure the performance of each model using both mean squared error (MSE) and mean absolute percentage error (MAPE).

MSE is a widely used regression evaluation metric in the cascade prediction domain. Let $y_k$ and $\widehat{y_k}$ represent the true value $\Delta R_T^k$ and predicted value $\widehat{\Delta R_T^k}$ logarithm, respectively, for the k-th cascade. Then, the calculation formula for MSE is as follows:

$$MSE={\displaystyle \frac{1}{K}}\sum _{i=1}^K{\left(y_i-\widehat{y_i}\right)}^2,$$

(29)

where K represents the total number of cascades, $y_k=log(\Delta R_T^k)$, $\widehat{y_k}=log(\widehat{\Delta R_T^k})$, and the lower the final MSE result, the better the model performance.

MAPE is used to measure the mean absolute percentage error between predicted values and actual values. It can reflect the percentage relationship between the prediction error and the actual value. Moreover, MAPE ignores the positive and negative nature of errors, directly examines the absolute value of errors, and can more objectively reflect predictive accuracy. The calculation formula of MAPE is as follows:

$$MAPE={\displaystyle \frac{1}{K}}\sum _{i=1}^K{\displaystyle \frac{\left|y_i-\widehat{y_i}\right|}{y_i}}.$$

(30)

For model parameters, the number of units per GRU hidden layer is 64, the initial learning rate for user embeddings and other variables is 0.0003, and the Adam optimizer is used for iterations. According to the DeepCas setting, the size of user embeddings is set to 50, the number of fully connected layers is 2, and the dimensions of the hidden layers are 64 and 32, respectively. The number of Transformer encoder layers is set to 2, and the number of heads is set to 4. N is the total number of sequences traversed in a cascade graph and M is the maximum sequence length in the traversal of the cascade graph, for the Sina Weibo dataset, if the observation time is 1 h, then $M=9$, if the observation time is 2 h and 3 h, then M = 10; for the twitter dataset, the observation time is 1 day or 2 days, $M=10$.

4.4. Performance Comparison

The overall predictive performance of all competitive methods on two different datasets is shown in

Table 2. The overall predictive performance of all competitive methods on two different datasets.

Model	Sina Weibo						Twitter
Observation Time	1 h		2 h		3 h		1 day		2 day
	MSE	MAPE	MSE	MAPE	MSE	MAPE	MSE	MAPE	MSE	MAPE
Feature-Linear	3.655	0.322	3.211	0.276	3.123	0.271	9.326	0.520	6.758	0.459
DeepCas	3.468	0.311	2.899	0.256	2.698	0.233	7.438	0.485	6.357	0.500
DeepHawkes	3.338	0.306	2.721	0.241	2.392	0.211	7.216	0.587	5.788	0.536
CasCN	2.976	0.266	2.643	0.232	2.376	$0.201$	7.183	0.547	5.561	0.525
GTGCN	2.421	0.227	2.378	$0.218$	2.322	0.215	6.988	0.472	5.172	0.377
CasFLow	2.289	0.212	2.198	0.222	2.278	0.234	6.954	0.455	5.143	0.361
CasHAN	2.287	0.211	2.193	0.221	2.275	0.272	6.999	0.459	5.132	0.356
CasNS-node	2.419	0.221	2.210	0.220	2.318	0.239	6.977	0.452	5.134	0.362
CasNS-sequence	2.491	0.226	2.326	0.221	2.351	0.242	6.982	0.442	5.156	0.353
CasNS	$2.216$	$0.194$	$2.174$	0.223	$2.263$	0.249	$6.943$	$0.433$	$5.112$	$0.332$

. Our proposed model shows better loss than other baseline methods on all datasets. For example, on the Sina Weibo dataset, when the observation time $T=1h$, the MSE predictive error of our model is 2.216, while the other seven methods increase by 0.071, 0.073, 0.205, 0.700, 1.122, 1.252, and 1.439, respectively. Among the seven methods mentioned above, Feature-linear exhibits the worst performance. This is because the predictive power of this model relies on the quality of handcrafted features, which require expertise from multiple domains. However, the design of these features often fails to effectively integrate with cascade graphs, greatly limiting their performance.

DeepCas introduced deep learning for cascade prediction, leading to significant performance improvements compared to traditional machine learning methods. The main reason for this improvement is that deep learning methods can autonomously learn feature representations of cascade graphs, eliminating the cumbersome process of manual feature extraction. However, due to its approach of aggregating sequence features into cascade features, its performance is still inferior to other deep learning-based methods. DeepHawkes integrates models based on self-exciting point processes, proposes improvements to the deficiencies of DeepCas, introduces time weight parameters, and improves predictive performance while increasing the interpretability of deep learning models. However, since the model can only obtain the embedding relation of neighboring nodes, the model’s performance cannot be qualitatively improved relative to DeepCas.

For the two advanced methods, GTGCN and CasFlow, their predictive errors on the two datasets are relatively close to each other and lower than the above four methods. Compared with DeepHawkes, GTGCN’s focus is on capturing the temporal evolution information within a snapshot and the dependencies between discrete graph sequences in the information diffusion process, which also improves predictive performance. The focus of CasFlow lies in obtaining high-quality initial embeddings for nodes. However, the predictive errors of GTGCN and CasFlow are still at least 9.2% and 3.2% higher than our model in MSE metrics. Compared with GTGCN and CasFlow, our model learns effective local cascade graph structure representations, obtaining high-quality node embeddings and embedding relationships with other nodes in the cascade graph, rather than learning complex global propagation graphs or high-quality node features, thus providing a greater contribution to cascade prediction.

For the state-of-the-art model CasHAN, our model still outperforms it on various datasets. Although CasHAN introduces community detection and inter-node self-attention mechanisms compared with other baseline models, to some extent, eliminating the influence of redundant nodes during the prediction process, it fails to capture the complete hidden relationships between the target node and all nodes in the sequence due to the use of dot product-based softmax for obtaining inter-node embeddings. However, our model improves upon this by simultaneously capturing the comprehensive hidden relationships between the target node and all nodes in the sequence using Query, Key, and Value, resulting in updated node embeddings that contain richer and more important information.

CasNS comprehensively considers the feature relationships between nodes and the features of the entire graph. The results shown in Table 2 prove the effectiveness of the proposed model. In addition, our model performs relatively well when using different observation times T. The larger the observation time T, the lower the MSE, and the easier the cascade prediction. After all, the size of a cascade graph is finite. The longer the observation time, the more cascade graph information is obtained, which is more conducive to predicting the popularity of information spread. In addition, we also found that when the observation time is small, our model can perform much better than other models, which proves that our model has very strong predictive power for incomplete cascades.

When comparing our experimental results on two real datasets, the error on the Twitter dataset is noticeably higher than the error on the Sina Weibo dataset. The specific reason is that compared with the Sina Weibo dataset, the Twitter dataset does not have originating nodes; in other words, the cascade graphs in the Twitter dataset only capture part of the path of information propagation, resulting in incompleteness of cascade graphs and increasing unpredictability, making predicting cascade increments more complex.

4.5. Ablation Study

Here are two variants of CasNS are proposed to prove the effectiveness of the two main components of CasNS:

CasNS-node: The model includes a node-level attention mechanism based on user influence but does not directly extract sequence features across different dimensions. It only utilizes the node embedding of the last GRU to represent the embedding of the entire sequence.

CasNS-sequence: The model directly captures sequence features at a meso-level across different dimensions. It removes the usage of the node-level self-attention mechanism at a micro-level to capture relationships between nodes.

Table 2 shows the predictive performance of these two CasNS variants. The loss of CasNS-node is lower than the loss of CasNS-sequence, indicating that the importance of node features in cascade prediction is relatively higher than that of sequence features. However, both models have lower losses compared with most of the baseline models, but higher than CasNS. This suggests that both node features and sequence features are essential in cascade prediction, and they complement each other. The ablation study demonstrates the interpretability of our model in capturing both the node features and sequence features of the cascade graph simultaneously.

4.6. Analysis of Sensitivity to Hyper-Parameters

4.6.1. The Number of Heads in the Multi-Head Attention Models

The number of heads in the learning meso-level features module to the number of attention heads used in the multi-head self-attention mechanism. To verify the impact of the number of heads in the multi-head attention models on model performance, we use the Sina Weibo dataset, set the observation time to 1, 2, and 3 h, set the number of units per GRU hidden layer to 64, the initial learning rate for user embeddings and other variables is 0.0003, and Adam optimizer is used for iterations, the size of user embeddings is set to 50, the number of fully connected layers is 2, the dimensions of the hidden layers are 64 and 32, respectively. The number of Transformer encoder layers is set to 1, and the number of heads N in the multi-head attention models must be divisible by the embedding dimension, so it is separately set to 1, 2, 4, 8, and 16.

Figure 4 shows the performance when N is 1, 2, 4, 8, and 16 on the Sina Weibo dataset. We can find that when N is not more than 4, increasing the number of heads can improve the model’s performance. For tasks that model text sequences, increasing the number of heads may achieve better results. More heads allow the model to pay better attention to different parts of the input sequence, improving the model’s representation and generalization abilities.

However, increasing the number of heads also increases the computational complexity and number of parameters of the model, thereby reducing the training and inference efficiency of the model, and increasing training time and computational cost. Therefore, when N equals 8, the performance will decrease instead. So a balance needs to be struck between computational resources and efficiency.

4.6.2. The Number of Sub-Encoder-Layers in the Encoder

The number of layers in the Learning meso-level features module refers to the deep neural network formed by stacking multiple self-attention layers and feed-forward neural network layers together. To verify the impact of the number of sub-encoder layers in the encoder on model performance, we still use the Sina Weibo dataset, keeping the other hyper-parameters unchanged, setting the number of heads N to 4, and the number of encoder layers to 1, 2, 3, 4 and 5, respectively.

Figure 5 shows the performance when num-encoder-layers are 1, 2, 3, 4, and 5 on the Sina Weibo dataset. We can find that for the same observation time, the performance is best when the number of layers is 2.

In theory, increasing the number of encoder layers can enhance the model’s expression ability and improve model performance within a certain range by repeatedly convolving the entire sequence information, thereby continuously improving model performance. However, in practice, the improvement becomes smaller and smaller with the increase in layers. This is because, for time series data, the number of layers is sufficient to make the contained historical content rich enough. Additional layers have limited returns, and too many layers will make model training more difficult, increase computational costs, and parameter convergence issues. This is because gradients propagate through too many layers, making it easy for gradients to diffuse or vanish.

4.7. Computational Complexity and Scalability

To provide a comprehensive evaluation of our proposed architecture, we address its computational complexity and scalability. For a concrete, quantitative perspective, we analyzed the model’s performance on our experimental setup, which consists of an Intel Core i5-13600KF CPU, 32 GB of RAM, and an NVIDIA RTX 3090 GPU (24 GB GDDR6X VRAM). The estimated performance metrics are summarized and compared against a lightweight baseline such as DeepCas in

Table 3. Model performance comparison.

Model	Training Time/Epoch (Minutes)	Inference Latency/Cascade (ms)	F1-Score
DeepCas	∼12 min	∼7 ms	0.893
Our Model	∼40 min	∼24 ms	0.928 ($+3.5\%$)

.

As the Table 3 illustrates, CasNS has approximately four times the parameters and requires 3–4 times longer training time per epoch compared to DeepCas. This is an expected outcome, as the model’s overall complexity is dominated by the Transformer’s self-attention mechanism, which has a complexity of $O(L^2·d)$. In contrast, the Capsule Network and GRU components, similar to the baseline, operate with a complexity that is linear with respect to the sequence length L.

This complexity profile represents a deliberate design trade-off, reflected in higher training and inference times. However, this is justified by a 3.5% increase in performance. We contend that this increase in computational time is an acceptable trade-off for the superior performance achieved. This higher computational cost is intentionally accepted to leverage the synergistic strengths of our hybrid model, which uniquely captures long-range dependencies, hierarchical features, and temporal dynamics. As supported by our experiments, the significant and consistent reduction in MSE validates this approach, demonstrating a meaningful improvement that justifies the additional computational investment in application contexts where accuracy is critical.

Moreover, it is noteworthy that the single-cascade inference latency remains in the millisecond range (approx. 15–20 ms), indicating its viability for near-real-time or offline prediction tasks. Nevertheless, we acknowledge that the quadratic complexity poses a scalability challenge for extremely long sequences. For practical large-scale deployment, this can be managed by applying sequence truncation to a feasible length $L_{max}$, a common practice we adopted in our experiments. Furthermore, a promising direction for future work is to integrate more efficient Transformer variants, such as those with linear complexity, which would substantially enhance scalability while preserving the core architectural benefits.

5. Conclusions

This paper proposes a framework called CasNS (Node-level and Sequence-level Features for Cascade Prediction) for predicting the future popularity of messages in social networks. We consider the information propagation paths in social networks as a directed graph, and our model effectively captures the micro-level node features, meso-level sequence features, and macro-level graph features in the directed graph. Experimental results on various real-world datasets demonstrate that all these features are essential for accurate information propagation prediction, and our model achieves state-of-the-art accuracy while effectively capturing these features. Furthermore, our model is highly scalable and can analyze more complex data. In future work, we will explore methods that can simultaneously capture node features, sequence features, and graph features more comprehensively. We aim to reduce the algorithm complexity and time complexity while ensuring accurate feature capturing.

Practical Implications and Future Work

In recent years, with the rapid proliferation of digital services and online communities, social network security has emerged as a paramount global challenge. Against this backdrop, the ability to accurately predict and effectively manage the dynamics of information propagation is crucial for maintaining the safety and stability of cyberspace.

The CasNS framework proposed in this paper offers a novel approach to addressing this challenge. Its core practical value lies in its strong performance in early-stage cascade prediction. The early and reliable identification of cascade growth trends is critical for a range of time-sensitive applications. Its most vital application is in the domain of social network security, where it can serve as a powerful tool for the proactive detection and mitigation of harmful misinformation, online rumors, and malicious content, thereby providing platform administrators and regulators with a valuable window for timely intervention. Moreover, the framework demonstrates significant potential in other areas, such as optimizing budget allocation in digital marketing and guiding public communication during public health emergencies. Technically, by providing reliable early predictions, CasNS offers decision-makers a critical opportunity for proactive intervention. The framework’s architecture is also designed with scalability in mind, allowing it to be extended to accommodate more complex propagation data and heterogeneous network structures.

Looking ahead, our future work will proceed along two primary avenues. First, we plan to further enhance the modeling of multi-level heterogeneous features within the CasNS framework to capture more nuanced propagation dynamics. Second, we will explore more efficient model architectures to significantly reduce computational and time complexity. Collectively, these efforts will aim to facilitate the effective deployment and application of CasNS in large-scale, real-world systems.