Analyzing the Properties of Graph Neural Networks with Evolutionary Algorithms

Abstract

Graph Neural Networks (GNNs) have achieved remarkable success in graph structure learning, but recent research reveals their vulnerability to carefully designed perturbations. To address this, ProGNN, a GNN model based on graph properties, was proposed to clean perturbed graphs by jointly learning robust GNNs and structural graphs. However, ProGNN’s performance is affected by the quality and distribution of input graph data, making it sensitive to specific types or distributions and underperforming with diverse data. This work introduces EProGNN, a novel GNN model incorporating an evolutionary algorithm, which jointly processes perturbed graph properties and introduces an evolutionary approach during graph structure updates. EProGNN generates diverse graph representations (individuals) through mutation functions and evaluates their quality using a fitness function, retaining high-scoring individuals for optimization and eliminating low-scoring ones. EProGNN overcomes overfitting, effectively handles perturbed graphs, and learns better graph embeddings, offering a novel method for graph structure optimization. Experiments on benchmark datasets and baselines demonstrate EProGNN’s effectiveness and the advantages of evolutionary algorithms in graph optimization. Experiments on benchmark datasets and baselines demonstrate EProGNN’s effectiveness and the advantages of evolutionary algorithms in graph optimization. For example, on the Cora dataset, EProGNN achieved a 1.48% higher accuracy than ProGNN at a 25% perturbation rate and demonstrated greater robustness against targeted attacks.

1. Introduction

Recent research have found that GNNs [1] have achieved breakthrough performance in many popular research fields, such as social network systems [2], financial transactions [3], biomedicine [4], knowledge graphs [5] and so on [6,7,8,9]. GNN combines graph theory with deep learning and can effectively learn relevant information from structured graph data [10,11]. The core task of GNN aims to learn the representation of nodes, that is, to map each node into a low-dimensional vector space. This node representation can reflect the characteristics of the node and its role in the graph. Due to its excellent performance, GNN has also achieved remarkable success in many downstream tasks, such as relationship extraction, graph generation [12], graph classification [13], subgraph matching [14] and so on [15,16,17,18].

Although GNN has achieved remarkable performance in many fields and downstream tasks, recent research shows that GNN is easily disturbed by the perturbation graph and becomes vulnerable [19]. GNN learns the embedding representation of graph nodes through message passing in the graph. This information transfer is accomplished by iteratively updating the node representation of the graph data, with each iteration involving information about neighboring nodes. If there are perturbed nodes, the erroneous information they convey may propagate in subsequent layers, thus affecting the representation of the entire graph. Furthermore, GNN is sensitive to the neighbor nodes of any node in the graph, which means the representation of a node is directly affected by the representation of its neighbor nodes. If a node’s neighbor nodes are modified, the node’s representation may be greatly affected, resulting in increased sensitivity of the model to the input graph. In addition, GNN is also sensitive to graph structure [20], which means that changes in the topology of the graph may lead to significant changes in node representation. If the perturbation adds new edges or removes the original set of edges, the adaptability of GNN to this structural change will decrease.

To solve the above problems, many researchers have provided different solutions. Adding adversarial perturbations to GNN [21] is a common method, in which this model is trained by adding adversarial perturbations to the input graph to make the model better adapt to various inputs. This approach is typically implemented by adding an adversarial loss function to the training objective. In addition, research shows that network distillation can also improve the stability of GNN [22]. For example, in the interaction network between teachers and students, a teacher network is trained to generate soft targets, and then these soft targets are used to train the student network to enhance the generalization performance and robustness of the model. At the same time, introducing regularization terms such as dropout into GNN [23] can also improve the robustness of the model. In addition, some researchers have proven that by processing graph properties, the anti-interference ability of GNN in perturbed graphs can also be solved [24]. Projecting the graph’s adjacency matrix or node representation into a low-rank space in GNN can help the model better adapt to missing information or uncertainty, thereby increasing its resistance to perturbations. For example, by sparsifying the adjacency matrix, the model can focus more on the core structure of the graph. Introducing feature smoothing into node representation, which means averaging or pooling the features of a node by considering its neighbor nodes in the graph structure, can help mitigate the noise or perturbation of node features. By processing the perturbed graph properties, GNN can learn better graph embeddings and provide effective solutions to vulnerabilities.

Although optimizing the properties of the perturbation graph can make the model robust, studies have found that this approach can cause ProGNN to fall into overfitting problems during the training process [25]. In some downstream tasks such as connection prediction, ProGNN needs to capture complex relationships in graph-structured data, and simplification of graph properties will lead to insufficient model capacity and thus cannot fit the graph-structured training data well. Therefore, when dealing with complex downstream tasks, oversimplifying ProGNN structure will cause overfitting problems. In addition, when there is insufficient training data, ProGNN stores a lot of noise and sample information from the training set and cannot generalize to unseen data. Even after graph properties are processed, ProGNN may still overfit on small-scale training data. Crucially, ProGNN relies on alternating optimization to satisfy strict constraints (e.g., low rank and sparsity). This gradient-based approach makes the model highly sensitive to initialization and hyperparameters, often leading to convergence to local optima. Therefore, ProGNN struggles to effectively explore the global graph structure space, limiting its ability to recover the true underlying topology in the presence of significant perturbations. This limitation necessitates a method capable of global search, such as the evolutionary strategy proposed in this paper.

To address this key challenge, this paper proposed the EProGNN model. This model solves the overfitting problem by adding an evolutionary algorithm and improves the model’s robustness and generalization ability. After EProGNN processes the graph properties, it uses three different mutation functions. Each mutation function generates a new graph structure. These three new graph structures are called individuals. Therefore, EProGNN presents a certain degree of diversity in the update process of the graph structure, allowing the model to try different graph structures instead of just relying on a single structure. In other words, the diversity of graph structures that EProGNN can learn has been improved. In addition, the evaluation and elimination mechanism is the core idea of the evolutionary algorithm. EProGNN uses this mechanism to evaluate and select graph structures. Different individuals will have different adaptability scores. Individuals with high scores will be retained, while individuals with low scores will be eliminated. By selecting individuals with better adaptability, the model can learn graph representation embeddings with greater generalization capabilities. This evaluation and elimination strategy effectively prevents overfitting because it prevents the model from simply overfitting the noise in the training data. At the same time, evolutionary algorithms continuously generate, evaluate, select, and eliminate individuals in multiple iterations. Through multiple iterations, EProGNN can find better solutions in the search space of the graph structure and continuously optimize the performance of the model during the update process of the graph structure. This approach helps this model generalize better across the entire graph structure space.

To sum up, the contributions of this paper are summarized in three aspects:

(1)

The model proposed in this paper uses an evolutionary algorithm to learn GNN parameters from the perturbation graphs. The goal of this model aims to utilize the graph properties in the perturbed graph to learn better graph node representation embeddings through the mechanism of evolutionary algorithm. The core principle of an evolutionary algorithm is to retain the optimal graph learning embedding parameters.

(2)

This is the first time that a novel framework model is proposed using an evolutionary algorithm in graph structure optimization tasks based on graph properties. This model continuously evolves and selects the optimal graph structure through the mutation, evaluation and selection of evolutionary algorithm. By evolving the graph structure, the model can adapt to different graph structures during the learning process and better capture the common features between graphs, thereby improving the generalization ability on graphs.

(3)

Experiments and tests on multiple challenging datasets demonstrated that EProGNN achieved convincing performance. In addition, EProGNN can adjust parameters more flexibly, making the learned graph embedding more adaptable to the requirements of the task.

The remainder of this paper is organized as follows. In Section 2, related work on graph neural networks and properties of graph neural networks are introduced. In Section 3, the problem statement is introduced. In Section 4, the methodological part of the model proposed in this paper is introduced. In Section 5, the performance of the model is analyzed and evaluated through experiments. In Section 6, this paper summarizes the contributions of EProGNN and future work.

2. Related Work

According to the research focus of this paper, the following briefly introduces the related work of GNNs and research on perturbation graphs.

2.1. Graph Neural Networks

Many real-world data are presented as graphs, such as social networks, knowledge graphs, bioinformatics and so on. Unlike text and image data, the data structure of graphs represents edges through the connection relationships between nodes. In addition, graph characteristics are related to nodes and edges and can include node properties, edge properties, and global properties. However, text and images are often characterized by word embeddings or pixel values. Therefore, traditional CNN [26] and LSTM [27] network models cannot handle graph data well. Recently, the introduction of GNN has solved this problem well and has become the most effective and outstanding tool for processing graph-structured data. GNN can adaptively learn the representation of nodes, is not limited by the input feature dimension, and can better adapt to the properties and connection relationships of different nodes. At the same time, GNN can simultaneously consider the relationship between a node and its neighbors, and has strong locality and globality. GNN updates the representation of each node by aggregating information from neighboring nodes to better capture the complex relationships in the entire graph structure. Naturally, many researchers have conducted extensive research on GNN and proposed more solutions based on it. To capture structural information more easily, SeHGNN [28] pre-computes neighbor aggregation using a lightweight mean aggregator, which reduces the complexity by removing the overused neighbor attention mechanism and avoiding repeated neighbor aggregation in each training cycle. To address the problem that the predicted categories of some nodes oscillate during GNN training, GRN [29] aims to reduce the prediction variance by iteratively optimizing the predictions of unstable nodes. In the prediction phase, GRN trains a graph dense encoder to predict node categories. In the relearning phase, GRN focuses on unstable nodes to optimize predictions. GraphPrompt [30] not only unifies pre-training tasks and downstream tasks into a common task template, but also adopts learnable prompts to help downstream tasks locate the most relevant knowledge from the pre-training model in a task-specific manner. IGNet [31] first applies a multi-scale extractor to obtain shallow features and uses them as the necessary input for building a graph structure. Then, the graph interaction module constructs the extracted intermediate features into a graph structure. At the same time, the graph structures of the two branches interact for cross-modal and semantic learning to improve the performance of downstream tasks. The proposal of these improved models further improves the robustness and applicability of GNN.

Some researchers have thought and studied the properties of graphs and proposed ProGNN [32]. This model works by decomposing the adjacency matrix representation of the original graph structure input into low-rank and sparse parts. The low-rank representation captures some global structural information of the graph, while the sparse representation contains some local perturbations or noise. By decomposing the adjacency matrix into low-rank and sparse parts, the structural information of the graph can be better captured. In addition, ProGNN smoothes node features to reduce feature noise or instability. The goal of feature smoothing aims to make the representations of adjacent nodes closer in the feature space to reduce the difference between representations. This can be achieved by taking into account the representation of neighbor nodes during the node update process. This helps improve the ProGNN model’s representation learning of nodes in the graph.

Generally, graph neural networks can be divided into spectral-based methods and spatial-based methods. Spectral-based methods use the graph Laplacian operator to define graph convolution in the spectral domain, while spatial-based methods perform convolution directly on the graph by aggregating neighbor information. This paper employs a Graph Convolutional Network (GCN) as the backbone encoder for the node classification task. This choice allows us to focus on optimizing the graph structure itself, which is consistent with the architecture of the baseline model ProGNN.

2.2. Research on Perturbation Graphs

Perturbed graphs refer to generating a changed graph structure by modifying or disturbing the original graph to a certain extent. If the perturbation introduces noise or destroys the structural information of the graph, it may cause the model performance to degrade [33]. For example, if key nodes or connections are removed, the model cannot properly capture the topology of the graph, affecting the accuracy of node classification or connection predictions. In recent years, researchers have conducted extensive research on how to better mitigate the negative impact of perturbation graphs on models. Some research works focus on how to design robust graph pooling operations to mitigate the effects of noise and perturbation [34]. This includes using robust graph pooling strategies, such as median pooling, to improve the model’s robustness to perturbed graphs. In addition, some researchers have proposed a normalization method based on the adjacency matrix, which improves the robustness of the GCN model to changes in the graph structure by considering implicit influence information [35]. This method shows better performance on graph structure perturbations. Self-supervised learning, meanwhile, is an unsupervised learning method in which a model is trained by solving tasks of its own creation. Research shows that self-supervised learning can help improve the model’s robustness to perturbation graphs [36]. In addition, GMNN [37] introduces the idea of Markov chain, which makes the model more robust to small changes in the graph structure by designing the transition probability on the graph. ARMA-GCN [38] combines the ideas of autoregression and sliding average, and designs an adaptive adjacency matrix smoothing strategy to alleviate the disturbance impact of the graph structure. Through adaptive smoothing operations, the ARMA-GCN model performs more robustly when processing disturbance graphs. Similarly, R-GAE [39] uses adversarial training to generate adversarial adjacency matrices and combines reconstruction error and adversarial loss for training. This method improves the model’s robustness to perturbed graphs through adversarial training, while maintaining good reconstruction capabilities for the graph structure. Naturally, GraphCL [40] utilizes contrastive learning of graphs to generate meaningful node representations and trains the model through self-supervised learning tasks. Through comparative learning, the GraphCL model can learn a more robust graph representation, which helps improve the model’s adaptability to disturbances.

3. Problem Statement

Graph data contains both features and structure. Once the structure is attacked, GNNs based on neighbor aggregation can experience a chain reaction of error propagation, leading to security vulnerabilities (such as failure in financial fraud detection). This is not just a matter of decreased model performance, but a significant security vulnerability in practical applications.

Before presenting the problem statement of the model in this paper, it is necessary to introduce some basic concepts and formula expression symbols. The Frobenius norm of the matrix M is defined as ${∥M∥}_F=\sqrt{\left(\sum i\sum j{M}_{ij}^2\right)}$. In the same way, the $l1$ norm of matrix M is $\left.{∥M∥}_1=\sum i\sum j|M_{ij}\right|$. The nuclear norm of matrix M is ${∥M∥}_{\ast }={\displaystyle \sum _{i=1}^{sort\left(M\right)}}{\xi }_i$, where ${\xi }_i$ represents the i-th singular value in matrix M. ${\left(M\right)}_{+}$ represents the positive value in matrix M. In the same way, ${\left(M\right)}_0$ represents the 0 value in the matrix M. $s\left(M\right)$ represents the sign matrix of matrix M, where $s{\left(M\right)}_{ij}\in [1,0,-1]$. $odot$ represents the element-wise product of two or more matrices. This paper uses $trace\left(M\right)={\sum }_i{M}_{ii}$ to calculate the trace of matrix M.

This article uses $\mathcal{G}=\left(\mathcal{V},\mathcal{E}\right)$ to represent a graph structure, Where $\mathcal{V}$ is the set of all nodes N in the graph $\left\{{\nu }_1,{\nu }_2,{\nu }_3,\dots ,{\nu }_k\right\}$. $\mathcal{E}$ represents the set of all edges in a graph $\left\{e_1,e_2,e_3,\dots ,e_n\right\}$. The set of edges between all nodes in the graph can also be expressed as an adjacency matrix $A\in {\mathbb{R}}^{N\times N}$. The adjacency matrix A can not only describe the connection between various nodes in the graph, but can also be used to analyze various properties of the graph. Usually the adjacency matrix $A_{ij}$ represents the connection status between node ${\nu }_i$ and node ${\nu }_j$. If there is a connection between the two, $A_{ij}=1$ otherwise $A_{ij}=0$. In addition, the characteristic properties of all nodes in the graph can be represented by the characteristic matrix $X\in {\mathbb{R}}^{N\times m}$, Where N represents the number of nodes, and m represents the number of characteristic properties present in each node. That is, each node has its own characteristic property representation. The characteristic matrix is defined as $X=\left[x_1,x_2,x_3,\dots ,x_N\right]\in {\mathbb{R}}^{N\times m}$. The feature matrix X enables the attribute information and connection relationships of each node to be disseminated and its message passed in GNN. In addition, the feature matrix X can also learn node embeddings in the feature space. Therefore, a graph can also be represented by $\mathcal{G}=\left\{A,X\right\}$. In the node classification task, only a part of the nodes in a graph are unlabeled nodes, while the remaining nodes are labeled nodes. In supervised learning tasks, labeled nodes are used to train the model, while unlabeled nodes are used for prediction. Assuming that there are e labeled nodes among N nodes, then the label set can be defined as $Y_{Le}=\left\{y_1,y_2,y_3,\dots ,y_e\right\}$. where $y_e$ represents the label of $v_e$.

Given a graph structure $\mathcal{G}=\left\{A,X\right\}$ and a marked node set $Y_{Le}$, the goal of this model in the training phase aims to learn a function $f_{\theta }:Y_{Le}\leftarrow {\mathcal{V}}_L$. Once $f_{\theta }$ is trained, it can be used for downstream tasks such as node classification. EProGNN can utilize the information of these label sets to predict the labels of unlabeled nodes. This paper uses softmax and other activation functions to output the probability distribution that each node belongs to different labels. For the labeled node set $Y_{Le}$, EProGNN calculates the gap between the real label and the predicted label by using a loss function such as the cross-entropy loss function. After this process is calculated, the parameters of the model are continuously updated through the backpropagation algorithm to minimize the loss. The loss function of EProGNN is defined as follows:

$${\mathcal{L}}_{{GNN}_{\theta }}=\sum _{v_i\in {\mathcal{V}}_L}l\left(f_{\theta }{(A,X)}_i,Y_i\right)$$

(1)

where $f_{\theta }{(A,X)}_i$ is the predicted value of graph node $v_i$. where $\theta$ is the parameter of the EProGNN objective function $f_{\theta }$. $f\left(l\right)$ is used to calculate the difference between the learned embedding and the real label. During the EProGNN training process, $f\left(l\right)$ selects the cross-entropy loss function.

4. The Proposed Framework

The original graph is easily affected by carefully designed perturbations. This paper calls the graph structure affected by such perturbations a poisoned graph. Research has found that poisoned graphs will significantly affect the original performance of ProGNN. The main reason for this phenomenon is that the poisoned graphs may have changed the original structure of the feature matrix, causing ProGNN’s feature smoothing processing to not be implemented well. In order to effectively resist graph perturbation, this article uses an evolutionary algorithm to solve the problem that matrix processing in ProGNN cannot be performed correctly. The framework diagram of the model proposed in this article is shown in Figure 1. The black edges are the original edges of the graph and the edges marked by red lines are perturbation edges maliciously added by graph perturbation. The significance of the existence of perturbation edges aims to greatly reduce the performance of downstream tasks. After the perturbed original graphs are input, the model will sequentially perform low-rank, sparse, and feature smoothing processing. The graphs after feature smoothing will enter the mutation operation. Each processed graph will obtain a different subgraph through three different mutation functions, and each subgraph will be regarded as a new individual. The model proposed in this paper will learn different objective functions based on new model parameters. The three objective functions obtained in each round are evaluated through the fitness function. The adaptability score refers to the degree of adaptation of an individual to the environment. The higher the score, the better the adaptability of the individual to the environment. In this paper, the individual refers to the objective function, and the objective function with a high adaptability score will be retained. On the contrary, the objective function with a low fitness score will be regarded as an individual with poor quality and will be eliminated. The entire evolution process will be iterated repeatedly until all clean graphs are updated.

Figure 1. The frame structure diagram of properties of GNNs with evolutionary algorithms.

4.1. Low Rank and Sparsity

Many real-world graphs are low-rank and sparse. For example, in a social network, there are certain rules in the connections between different users. Some users are only connected to a small number of users, but have no connection with the vast majority of users. This pattern is not special, but is common in the entire social network. In other words, graphs can be represented with less information. Some graph perturbations directly add connections at nodes where two feature properties are different. This will greatly increase the rank of the adjacency matrix, increase the noise sensitivity of the graph, and affect the performance of updating the graph structure. Low-rank and sparse processed graphs allow them to be represented in a more compact form, which reduces storage and structural complexity. Low-rank graphs can reduce the dimensionality of data while retaining the main structural features of the graph, thereby learning embedding vectors with low-rank structures. The way this paper handles low-rank and sparse graphs aims to use singular value decomposition. With a perturbed adjacency matrix $A\in {\mathbb{R}}^{m\times n}$, the calculation formula of its singular value ${\xi }_A$ is as follows:

$${\xi }_A=A{\displaystyle \frac{v_k}{u_k}}$$

(2)

where $v_k$ is all k column elements of the right singular vector matrix $A\in {\mathbb{R}}^{m\times n}$. In the same way, $u_k$ represents all k row elements of the left singular vector matrix $\mathcal{U}\in {\mathbb{R}}^{m\times m}$. It should be noted that both V and $\mathcal{U}$ are unitary matrices, that is, their elements are composed of orthonormal bases. V and U are obtained by calculating the eigenvalues of $A^{T}A$ and $A{A}^T$, respectively. Because the calculation process of eigenvalues and eigenvectors is very complex, specific function libraries such as Numpy and Scipy need to be used. Here is a simple formula to use the eigenvalue equation of $\mathcal{U}$: $det\left(A{A}^T-{\lambda }_{\mathcal{U}}I\right)=0$, where ${\lambda }_{\mathcal{U}}$ is the eigenvalue of the left singular vector matrix $\mathcal{U}$, and I is the identity matrix. The formula for constructing a new adjacency matrix is as follows:

$${A^{\prime }={\mathcal{V}}_i^{\prime T}{\mathcal{U}}_i^{\prime }diag{\left({\left({\xi }_i-\alpha \right)}_{+}\right)}_i}^{\prime }$$

(3)

where ${\mathcal{V}}_i^{\prime T}$ is the new right singular vector matrix after retaining the first i singular values, and similarly ${\mathcal{U}}_i^{\prime }$ is the new left singular vector matrix after retaining the first i singular values. ${diag{(·)}_i}^{\prime }$ is a new singular value matrix retaining the first i singular values. $diag{(·)}_i$ is a diagonal matrix, and the elements on the diagonal are composed of the first i singular values in descending order. The matrix M contains two regular terms, $l1$ norm and nuclear norm, and both regular terms are non-differentiable. The process of finding the optimal objective function of matrix M requires optimizing these two regular terms. The model proposed in this paper first uses the gradient strategy at each step, and then uses proximal operators to deal with non-differentiable regular terms. The formula for this process is as follows:

$$M^{\left(i\right)}={\mathcal{P}}_{\kappa R}\left(\kappa {\nabla }_M\mathcal{L}(M,A){+M}^{(i-1)}\right)$$

(4)

where $\kappa$ is the learning rate and ${\mathcal{P}}_R$ is the proximal operator function. The specific calculation method of this function is as follows:

$${\mathcal{P}}_R\left(E\right)={argmin}_{M\in {\mathbb{R}}^{N\times N}}^{\frac{1}{2}}{∥M-E∥}_F^2+R\left(M\right)$$

(5)

The calculation formulas of l1 norm and nuclear norm are as follows:

$${\mathcal{P}}_{\beta {\parallel M\parallel }_1}\left(E\right)={\left(\left|E\right|-\beta \right)}_{+}⨀s\left(E\right)$$

(6)

$${\mathcal{P}}_{\alpha {\parallel M\parallel }_{\ast }}\left(E\right)=\mathcal{V}\mathcal{U}diag{\left({\left({\xi }_i-\alpha \right)}_{+}\right)}_i$$

(7)

In order to simultaneously optimize the objective function containing two non-differentiable regular terms, this model uses the incremental proximal descent algorithm. By utilizing this looping method, the matrix M is updated layer by layer. The specific formula is as follows:

$$\left\{\begin{array}{c}{M}^{\left(i\right)}=M^{(i-1)}-\kappa ·{\nabla }_{M}L(M,A),\\ \left.\hspace{1em}{M}^{\left(i\right)}={\mathcal{P}}_{\kappa {\parallel M\parallel }_{\ast }}{\left(M\right.}^{\left(i\right)}\right),\\ \left.\hspace{1em}{M}^{\left(i\right)}={\mathcal{P}}_{\kappa {\parallel M\parallel }_1}{\left(M\right.}^{\left(i\right)}\right).\end{array}\right.$$

(8)

4.2. Feature Smoothness

Some studies have proven that the characteristics of adjacent nodes in the graph have a certain degree of similarity. Through feature smoothing, the information of neighbor nodes can be integrated into the node’s own representation. The nodes after fusing feature information have smaller feature information differences, which helps better embedding learning. In addition, the feature smoothing of neighbor nodes helps to transfer and propagate node information in the graph. When nodes in the graph lack noise or labels, feature smoothing can better help nodes adapt to the local structure of the graph. Recent research shows that graph perturbation tends to connect nodes with different characteristic property information. This perturbation method will increase the dimension of the original feature matrix and destroy the feature property information of the original nodes. The method used in the model proposed in this paper is average features. Calculate the feature mean of all neighbor nodes of each node in the feature matrix X, and then combine the feature mean with the node itself. From this, the new feature matrix $X^{\prime }$ after smooth feature processing is obtained. The above process is shown in the following formula:

$${X_{ij}}^{\prime }={\displaystyle \frac{{\sum }_{k=1}^{n_i}{w}_{ik}{X}_{ik}}{n_i{\sum }_{k=1}^{n_i}\left.{\left(w\right.}_{ik}-\gamma \right)}}$$

(9)

where $w_{ik}$ refers to the relationship weight between the selected node i and its neighbor node k. The relationship weight aims to further measure the importance of different neighbor nodes to the current node. Each element in the new feature matrix ${X_{ij}}^{\prime }$ is the weighted average of the features in the corresponding i-th row of the original feature matrix $X_{ik}$. Such a weighted average can more finely control the influence of neighbor nodes on the current node representation to adapt to the relationship between different nodes.

4.3. Mutation

The entire evolutionary algorithm is mainly divided into three parts: mutation, evaluation, and selection. Since the selection step is relatively simple, this paper will not describe it in too much detail. This paper describes in detail the two more important parts of mutation and evaluation. The framework of the entire evolutionary algorithm is shown in Algorithm 1. The mutation step will allow the model proposed in this paper to learn three different mutation functions in order to continuously generate individuals. As mentioned above, this model performs low-rank sparse processing on the adjacency matrix, and the $A^{\prime }$ obtained after processing will learn the variation function to generate a new sub-matrix set $m\left(A^{\prime }\right)=\left\{{A^{\prime }}_1,{A^{\prime }}_2,{A^{\prime }}_3\right\}$. $m\left(A^{\prime }\right)$ will improve the diversity of graph structure learning and increase the robustness of the model, allowing the model to adapt to different graph structures. In the process of resisting graph perturbation, this process can change the nature of the original perturbation, allowing the graph structure to learn better embedding representations. That is, the original adjacency matrix may have been learned during the mutation process and the perturbed matrix has been learned during the training process. During each iteration, each original graph undergoes the same mutation steps to generate three different individuals. The following parts introduce these three different variograms in detail.

Random Walks Mutation: Assume that an adjacency matrix $A^{\prime }$ is input and is waiting for mutation. ${A^{\prime }}_{ij}$ represents whether there is an edge between node i and node j. Initialize a probability matrix ${\Pi }_i$, ${\Pi }_{ij}$ represents the transition probability from node i to node j. For each node i in the graph, perform a random walk process starting from i. At each step, the probability distribution of reaching neighbor nodes from the current node i is calculated through the probability matrix ${\Pi }_i$. Construct a transition matrix $\mathcal{P}$ based on the recorded walking path, where ${\mathcal{P}}_{ij}$ represents the transition probability from node i to node j. Since the uncertainty of the transfer between nodes and the weight of the edges need to be considered, a random matrix and a probability matrix need to be added to modify the transfer matrix. This process is represented by the following formula:

$${\mathcal{P}}^{\prime }=softmax\left(\mathcal{P}+\gamma R\right)·\Pi$$

(10)

where $\gamma$ is the parameter that controls randomness, $softmax$ is the activation function, R is the random matrix. The random matrix R is introduced to add controlled randomness to the transition probabilities, enhancing the algorithm’s exploration capabilities while preventing overfitting to the existing graph structure. The formula for constructing a new adjacency matrix is as follows:

$${A^{\prime }}_1={\displaystyle \frac{{\mathcal{P}}^{\prime }+{\left({\mathcal{P}}^{\prime }\right)}^T}{2}}$$

(11)

Algorithm 1 Evolutionary Algorithm
Input: GNNs population P, total training epochs $N_e$, parameters used in the evolution process $\left\{\lambda ,\nu \right\}$
Output: evolved population $S_e$
1:	Initialization: graph structures $\mathbb{G}=\left(\mathcal{V},\mathcal{E},x\right)$
2:	Mutate population into individuals according to Equations (11), (14) and (17)
3:	Evaluate various individuals according to Equation (18)
4:	while individuals cannot adapt to the environment do
5:	Input attacked graphs ${\mathbb{G}}^{\prime }=\left(\mathcal{V},{\mathcal{E}}^{\prime },x\right)$
6:	Mutate population into individuals according to Equations (11), (14) and (17)
7:	Evaluate various individuals according to Equation (18)
8:	Select the best individual
9:	end while
10:	$\mathbf{return}$ evolved population $S_e$

In the process of calculating ${A^{\prime }}_1$, its directionality needs to be considered, so the transfer matrix needs to be calculated to obtain the modified transposed form. The first matrix ${A^{\prime }}_1$ after the mutation operation can be obtained through the random walk method.

Betweenness Centrality Mutation: Betweenness centrality mutation generates a new matrix by calculating the influence of node i on other nodes j in the same graph structure. First, the betweenness centrality of a node is calculated, which is based on the number of shortest paths between the node and other nodes. The calculation formula of betweenness centrality ${\mathcal{C}}_{A^{\prime }}\left(v\right)$ is as follows:

$${\mathcal{C}}_{A^{\prime }}\left(v\right)=\sum _{s\ne v\ne t}{\displaystyle \frac{{\varrho }_{st}\left(v\right)}{{\varrho }_{st}}}$$

(12)

where ${\varrho }_{st}$ is the number of shortest paths from node s to node t in the adjacency matrix $A^{\prime }$. ${\varrho }_{st}\left(v\right)$ is the number of paths passing through node v in the shortest path. The purpose of the restriction of the inequality sign aims to ensure that node v is neither node s nor t. Next, the betweenness centrality needs to be normalized. Assume that among N nodes, all nodes are directly connected to all other nodes. The calculation formula of normalized betweenness centrality is as follows:

$${\mathcal{C}}_{A^{\prime }}^{norm}\left(v\right)={\displaystyle \frac{{\mathcal{C}}_{A^{\prime }}\left(v\right)}{(N-1)(N-2)/2}}$$

(13)

Through normalization processing, the value of ${\mathcal{C}}_{A^{\prime }}^{norm}\left(v\right)$ is limited to $[0, 1]$. This makes it easier to compare the betweenness centrality values of different networks. This can more clearly determine which nodes in the graph play a more important connecting role in the shortest paths to other nodes. Finally, the matrix weight ${\omega }_{ij}$ needs to be set, and the node update is determined through the threshold $\mathcal{T}$. The weight ${\omega }_{ij}$ calculation formula is as follows:

$${\omega }_{ij}={\displaystyle \frac{{\mathcal{C}}_{A^{\prime }}^{norm}\left(i\right)+{\mathcal{C}}_{A^{\prime }}^{norm}\left(j\right)}{2}}$$

(14)

The range of $\mathcal{T}$ is $[0, 1]$, and the specific numerical selection of this value in different downstream tasks is also different. The original matrix $A^{\prime }$ can be updated through the threshold $\mathcal{T}$, and finally ${A^{\prime }}_2$ is obtained.

Stochastic Block Mutation: Random block mutation is a method that uses a probabilistic graph model to reconstruct the connection relationship between pairs of adjacency matrix nodes. The nodes in the original graph will be divided into several different blocks using community detection algorithms, which group nodes based on their connection density. Nodes within each block are connected with a certain probability, while the connection probabilities between different blocks may be different. Random block mutation is divided into two steps: defining probability parameters and generating adjacency matrices. Assume that there are K node blocks, $\left\{n_1,n_2,\dots ,n_K\right\}$ represents the number of nodes in each node block. The calculation of the connection probability matrix $\mathcal{S}$ and the probability matrix $\mathcal{Q}$ within the node block is as follows:

$${\mathcal{S}}_{k,l}={\displaystyle \frac{E_{k,l}}{n_k·n_i}}$$

(15)

$${\mathcal{Q}}_{k,k}={\displaystyle \frac{E_{k,k}}{n_k\left(n_k-1\right)}}$$

(16)

where $E_{k,l}$ represents the number of connections between nodes in node block k and nodes in node block l. $E_{k,k}$ represents the number of node connections in node block k. For the new adjacency matrix ${A^{\prime }}_3$, the existence of edges can be judged through the following probability distribution:

$$P\left({{A^{\prime }}_3}_{ij}=1\right)=\left\{\begin{array}{c}{\mathcal{S}}_{k,l},ifi,j\in k,l\\ {\mathcal{Q}}_{k,k},otherwise.\end{array}\right.$$

(17)

Through the above formula, we can get the probability of an edge between any two nodes i and j in ${A^{\prime }}_3$, and then update the adjacency matrix. The advantage of using the connection probability matrix $\mathcal{S}$ and the node intra-block probability matrix $\mathcal{Q}$, respectively, is that the generated graph conforms to a predetermined probability distribution in the connection patterns between blocks and within blocks.

4.4. Evaluation

In the entire process of processing graph data by the model proposed in this paper, the evaluation section aims to evaluate the quality of different individuals through the fitness function. Only individuals with high fitness scores will be considered to be of high quality and thus selected and retained. After inputting a perturbation graph, this model will first perform low-rank feature smoothing processing, and then perform mutation operations. However, this two-stage approach may result in suboptimal performance of the GNN model on a given downstream task. Therefore, this paper proposes a new joint learning method to learn better GNN parameters. Therefore the fitness function F in the evaluation section is as follows:

$$F_{\mathcal{L}}=-\left[{∥A-\mathit{M}∥}_F^2+\alpha {∥M∥}_{\ast }+\beta {∥M∥}_1+\delta \sum _{v_i\in {\mathcal{V}}_L}l\left(f_{\theta }{(A,X)}_i,Y_i\right)\right]$$

(18)

The fitness function $F_{\mathcal{L}}$ consists of four key components: (1) a reconstruction error term that ensures the processed graph maintains similarity with the original structure, (2) a nuclear norm term that promotes low-rank properties and captures essential graph features, (3) an L1 regularization term that maintains graph sparsity, and (4) a task-specific loss term that measures the performance of the GNN model on downstream tasks. These components work together to balance structure preservation, feature extraction, and task performance.

Where $\delta$ is a predefined hyperparameter that controls the GNN model loss function. Since adversarial attacks will maximize ${\mathcal{L}}_{{GNN}_{\theta }}$, preprocessing functions such as low-rank sparse are needed to optimize the objective function. In the training stage, graph structures with higher accuracy in downstream tasks such as node classification will get higher $F_{\mathcal{L}}$. The graph structure with high $F_{\mathcal{L}}$ indicates that it will have better classification performance, and the graph structure will be regarded as the optimal individual and thus retained. On the contrary, graph structures with low $F_{\mathcal{L}}$ will be regarded as low-quality individuals and then eliminated.

4.5. Overview of EProGNN

Existing methods (such as ProGNN) are typically based on gradient-based optimization, which can easily get stuck in local optima. EProGNN’s innovation lies in introducing population-based search, utilizing evolutionary mechanisms to explore a wider range of graph structures.

After introducing the detailed steps of each part of EProGNN, Algorithm 2 presents the entire EProGNN training process. In line 1, the adjacency matrix, feature matrix and parameters of the perturbation graph input to EProGNN are first initialized. In lines 3 to 5, EProGNN will perform low-rank sparse and feature smoothing processing on the adjacency matrix and feature matrix, respectively. In line 7, the classifier of EProGNN is trained. In line 9, the graph structure after attribute processing will generate different new graph structures based on three different mutation functions. These new graph structures are called individuals. In line 10, different individuals are evaluated for individual quality by the fitness function. In lines 13 to 14, EProGNN will sort according to the fitness score, and finally filter out the individuals with the highest score in each epoch. Through this mechanism of evolutionary algorithm, EProGNN can find an optimal graph structure in each iterative training for subsequent downstream task training.

4.6. Complexity Analysis

This section provides a detailed analysis of the time complexity of EProGNN and the baseline ProGNN. First, let $\left|V\right|$ represent the number of nodes, $\left|E\right|$ represent the number of edges, d represent the feature dimension, and L represent the number of GNN layers. The time complexity of ProGNN mainly depends on the singular value decomposition (SVD) required for the low-rank constraints, which is approximately $O\left(\right|V{|}^3)$ in the worst case and $O\left(\right|V{|}^{2}k)$ in most cases. This makes ProGNN computationally expensive when processing large graph data. For EProGNN, its time complexity is determined by the evolutionary process. Let G be the number of model iterations and P be the population size. Its cost mainly consists of two parts: (1) fitness evaluation (GCN training) and (2) mutation operations. The complexity of one GCN training iteration is $O\left(L\right|E|d+L|V\left|d^2\right)$. Therefore, the total time complexity of EProGNN is approximately $O(G·P·(L\left|E\right|d+L\left|V\right|d^2\left)\right)$. Although introducing a population size P increases the total computational cost compared to training a single model, this cost is justified for two reasons:

• Parallelism: Unlike the gradient-based sequential updates in traditional ProGNN, the fitness evaluation of individuals in EProGNN is independent. Therefore, the complexity caused by the factor P can be effectively offset by parallel computation across multiple GPUs.
• Performance trade-offs: The additional computational overhead enables EProGNN to overcome the limitations of traditional ProGNN gradient descent, thereby escaping local optima and achieving greater robustness against targeted attacks, a primary goal for security-critical applications.

Algorithm 2 Properties GNNs with Evolutionary Algorithm
Input: Adjacency matrix of perturbation graph A, Attribute matrix of perturbation graph X, Label set $Y_{Le}$ of the perturbation graph, Hyperparameters $\alpha$, $\beta$, $\gamma$, $\delta$, Probability matrix $\Pi$.
Output: EProGNN model parameters sets $\left\{{\Phi }_1^{best},{\Phi }_2^{best},{\Phi }_3^{best},\dots ,{\Phi }_n^{best}\right\}$.
1:	Initialization: initialize basic perturbation graphs $\left\{A_1,A_2,A_3\right\}$, initialize basic perturbation attribute matrices $\left\{X_1,X_2,X_3\right\}$, initialize model parameters $\Phi$;
2:	while Stopping condition is not met do
3:	% Perform low-rank, sparse and feature smoothing processing on the perturbation graphs
4:	Low-rank and sparsify the input adjacency matrix according to Equations (3) and (8);
5:	Smooth the input feature matrix features according to Equation (9);
6:	for $j\leftarrow 1$ to i do
7:	% Training model classifier
8:	for $k\leftarrow 1$ to $i_{\theta }$ do
9:	The graphs after attribute processing are mutated through Equations (11), (14) and (17);
10:	Different individuals perform quality evaluation through Equation (18);
11:	end for
12:	end for
13:	$Sort\left\{F_{\mathcal{L}}^{j,k}\right\}$;
14:	$\left\{{\Phi }_1^{best},{\Phi }_2^{best},{\Phi }_3^{best},\dots ,{\Phi }_n^{best}\right\}\leftarrow \left\{{\Phi }_1^{j_1{k}_1},{\Phi }_1^{j_2{k}_2},{\Phi }_1^{j_3{k}_3},\dots ,{\Phi }_n^{j_i{j}_{i_{\theta }}}\right\}$
15:	end while
16:	return $\left\{{\Phi }_1^{best},{\Phi }_2^{best},{\Phi }_3^{best},\dots ,{\Phi }_n^{best}\right\}$

5. Experiments

In this section, this paper evaluate the effectiveness and feasibility of EProGNN against complex graph adversarial attacks. This section discusses the performance capabilities of EProGNN under different modes of adversarial attacks and a large amount of advanced experimental data through a large number of convincing experimental data. In addition, the performance changes of EProGNN under different attribute modes are further discussed. Before presenting the experimental results and comparative analysis in this paper, this paper first introduces the experimental setup.

5.1. Experimental Settings

5.1.1. Datasets

To evaluate the effectiveness of EProGNN, we conducted experiments on five benchmark datasets, including two social networks and three citation networks. The detailed statistics of these datasets are summarized in Table 1.

Table 1. Statistics of the five benchmark datasets used in the experiments.

Dataset	Type	Nodes	Edges	Classes
BlogCatalog	Social	10,312	333,982	39
Orkut	Social	48,760	98,020	-
Cora	Citation	2708	5429	7
Citeseer	Citation	3327	4732	6
Pubmed	Citation	19,717	44,338	3

• Social Networks: We utilize BlogCatalog [41] and Orkut [42]. BlogCatalog represents social relationships among bloggers, where node labels denote user interests derived from metadata. Orkut is a large-scale social network dataset where edges indicate friendships between users and groups.
• Citation Networks: We employ three standard citation graphs: Cora [43], Citeseer [44], and Pubmed [45]. In these graphs, nodes represent scientific publications, and edges denote citation links. For Cora and Citeseer, node features are binary word vectors indicating the presence of specific words. Pubmed involves diabetes-related publications, utilizing TF-IDF weighted word vectors as node features.

5.1.2. Baselines

In order to analyze the effectiveness and feasibility of EProGNN, this paper uses the adversarial attack repository DeepRobust [46] to compare and analyze it with the most representative and convincing GNN model and defense model, respectively. The relevant information of the baseline model used in this experiment is as follows:

(1)

GCN [47] is a method for learning and representing sets of vertices and edges in GNNs, which truly implements the mapping from words to graphs. Image inputs are successfully used as latent image outputs through node embedding and random walk strategies.

(2)

GAT processes graph-structured data. It aggregates neighbor nodes through the attention mechanism, realizes adaptive allocation of weights for different neighbors, and greatly improves the expressiveness of GNNs.

(3)

GraphSAGE is a popular graph neural network architecture for inductive node embeddings. The model exploits node features (e.g., text attributes, node configuration information, node degree) to learn an embedding function that generalizes to unseen nodes. By adding node features into the learning algorithm, the model simultaneously learns the topology structure of each node neighborhood and the distribution of node features in the neighborhood.

(4)

ProGNN is a model that can learn graph structure and GNN parameters at the same time. The goal of this model aims to defend against the most common adversarial attack setting on graph data, namely poisoning adversarial attacks on graph structures. In this setting, modifying the edges before training the GNN already disturbs the graph structure, while the node features of the model itself are not changed.

5.1.3. Parameter Settings

In this experiment, a small number of walk nodes are randomly selected for training on the input graph dataset. In this experiment, the number of nodes used for verification is consistent with the number of nodes used for training. The remaining nodes in each round of experiments are used for staged testing of the experiment. The data results of each experiment are analyzed and compared based on the completion of 50 iterations of training. For the input dimension of each layer of the GCN model, a series of parameter settings, such as the number of nodes, are all experimented with according to the scheme provided by the original author. In the verification experiment of the GAT model, under the premise of following the standard experiment mentioned by the original author, the preset values of the structure and loss or evaluation function are set to $dropout=0.0$ and $residual=True$, respectively. Conducting the experimental verification of GraphSAGE, this experiment sets $90\%$ of the nodes as training and $10\%$ of the nodes as unsupervised training for sufficient experiments. For the experimental verification part of RGCN, this experiment sets the range of the number of hidden nodes and non-hidden nodes from $\left\{16,32,64,128\right\}$ and $\left\{32,64,128,256\right\}$ respectively. For the GCN additional algorithm experiment part, this experiment sets the equivalent edge set threshold range and perturbation graph value range according to the relevant literature. The value range of the former is set to $\left\{0.01,0.03,0.05,0.10\right\}$, and the value range of the latter is set to $\left\{5,15,50,100,150,200\right\}$. Since the main task of this paper is node classification, and the samples in each category are relatively balanced, accuracy (%) is used as the main evaluation metric to measure the model’s resilience against adversarial attacks.

5.2. Defense Performance

In the experimental part of this paper, we prove the excellent performance of EProGNN in node classification from multiple perspectives. ‘-’ in Table 2 shows that insufficient memory is available for experimental comparison. The following three aspects will be elaborated next:

Table 2. Node classification performance (accuracy (%)) under non-targeted perturbation. The bolded text in the table indicates the optimal experimental results.

Dataset	Ptb Rate (%)	GCN	GAT	GraphSAGE	RGCN	GCN-Jaccard	GCN-SVD	Pro-GNN-fs	Pro-GNN	EProGNN
Cora	0	$79.50$	$79.85$	79.93	$78.56$	$77.11$	$76.44$	$79.12$	$79.02$	$79.38$
	5	$67.71$	79.13	$77.87$	$76.09$	$75.45$	$75.74$	$78.09$	$78.31$	$78.83$
	10	$57.90$	$74.42$	$72.90$	$70.12$	$68.52$	$70.14$	$74.98$	$74.12$	75.10
	15	$54.11$	$69.87$	$66.21$	$64.49$	$62.23$	$67.12$	$69.88$	$70.91$	71.42
	20	$51.71$	$66.12$	$60.13$	$60.09$	$59.81$	$60.11$	$67.45$	$67.15$	68.19
	25	$48.92$	$60.23$	$57.71$	$54.90$	$55.12$	$59.25$	$62.28$	$63.37$	64.85
Citeseer	0	$67.23$	$68.13$	$67.79$	$68.11$	$67.03$	$69.15$	69.88	$69.54$	$69.38$
	5	$65.71$	$66.13$	$65.24$	$66.90$	$66.23$	$67.21$	$68.52$	68.91	$68.03$
	10	$62.26$	$64.32$	$60.16$	$65.14$	$63.39$	$64.12$	$66.98$	$67.12$	67.42
	15	$59.91$	$62.25$	$58.73$	$63.25$	$57.92$	$60.15$	$66.14$	$66.34$	66.91
	20	$57.25$	$60.09$	$56.32$	$59.54$	$54.88$	$56.91$	$65.09$	$65.23$	65.36
	25	$51.67$	$57.25$	$56.11$	$58.12$	$53.81$	$51.56$	$60.11$	$60.01$	60.16
Polblogs	0	91.98	$90.78$	$91.09$	$90.24$	-	$90.65$	$91.24$	-	$91.75$
	5	$86.62$	$83.31$	$85.25$	$88.91$	-	$89.81$	91.09	-	$90.98$
	10	$80.24$	$79.90$	$71.24$	$81.14$	-	$78.92$	$90.87$	-	90.90
	15	$76.24$	$62.25$	$69.98$	$70.26$	-	$71.14$	$86.25$	-	87.81
	20	$71.11$	$61.09$	$62.78$	$63.24$	-	$69.42$	$84.54$	-	85.24
	25	$64.84$	$60.25$	$60.24$	$45.67$	-	$50.71$	$79.87$	-	80.26
Pubmed	0	$82.21$	$80.92$	-	$82.85$	$80.11$	$82.21$	83.98	$83.41$	$83.45$
	5	$82.14$	$79.78$	-	$82.25$	$79.45$	$81.21$	$83.24$	$83.34$	83.39
	10	$82.04$	$79.24$	-	$82.13$	$79.35$	$81.19$	$83.11$	$83.14$	83.24
	15	$81.78$	$79.57$	-	$82.09$	$79.21$	$80.87$	$82.26$	$83.11$	83.06
	20	$81.45$	$79.22$	-	$81.85$	$78.59$	$80.57$	$82.13$	$82.74$	82.87
	25	$80.93$	$78.01$	-	$81.76$	$78.13$	$80.24$	$82.05$	$82.24$	82.57
Orkut	0	$78.81$	$77.24$	-	$78.12$	$77.90$	$78.85$	$78.81$	78.98	$78.88$
	5	$74.81$	$75.87$	-	$74.68$	$75.82$	$76.57$	$77.03$	$77.09$	77.13
	10	$68.24$	$72.22$	-	$70.38$	$71.84$	$73.26$	$75.16$	$75.24$	75.92
	15	$67.71$	$70.57$	-	$66.25$	$70.26$	$70.91$	$73.18$	$73.01$	73.25
	20	$64.01$	$64.41$	-	$61.24$	$64.27$	$68.26$	$72.33$	$72.14$	72.45
	25	$61.27$	$61.09$	-	$60.15$	$60.91$	$61.24$	$70.28$	$70.13$	70.36

• Targeted Perturbation: The scheme aims to conduct a targeted single edge set perturbation on a specific node on the canonical graph structure in order to deceive the graph model on the original target node. This paper uses two targeted perturbation methods, poisoning perturbation and evasion perturbation, and this is currently the most advanced purposeful perturbation on graph structure model data. Under this perturbation scheme, the attacker may not be able to directly modify the target node, and may only be able to access some nodes other than the target node or access incomplete data. In doing so, the experimenter can clearly distinguish between the perturbation node and the target node.
• Targetless Indiscriminate Perturbation: This perturbation scheme is different from the former one. The indiscriminate perturbation is an attempt to reduce the overall performance of the experimental model on the entire graph so as to examine the ability of the model to resist comprehensive perturbations. Under the indiscriminate perturbation, the attacker tries to reduce the generalization ability of the model and reduce the performance on unlabeled nodes. The core idea of this perturbation scheme aims to use the adjacency matrix of the experimental graph structure as a hyperparameter. It is worth noting that the objective function of the total scheme is a bi-level problem, and each element in the adjacency matrix of the graph can only be 0 or 1, so this is also a discrete optimization problem.
• Randomization Perturbation: This perturbation scheme is different from the previous two. After the randomization perturbation selects a fixed perturbation graph, it is assumed that the selected graph structure is empty, and then the graph structure to be tested is randomly selected and randomly assigned. In other words, the perturbation scheme deletes random edges in any clean graph structure and fills in wrong edge sets. This kind of wrong filling can be completely regarded as a source of poisoning noise to some extent, and several such sources of poisoning noise will be added to each graph structure. According to the experimental data, the perturbation scheme is quite convincing and provable.

This paper first used various different kinds of perturbation methods to make canonical graphs under perturbation. This paper used the five most representative datasets to train EProGNN and various convincing baseline models on the graph models affected by the perturbation and analyzed the node classification performance obtained under these methods and datasets in detail.

5.2.1. Against with Targetless Indiscriminate Perturbation

This paper first used the canonical graph structure under different modes to conduct different degrees of benchmark experiments against non-target perturbations through different models and datasets. The perturbation method of perturbation aims to use the meta-gradient. This perturbation method directly regards the graph structure matrix under different paradigms as non-fixed hyperparameters, and then follows the special pattern of the graph structure to derive. This paper randomly uses a NIPA method to directly inject a large number of simulated nodes into the graph structure of different modes. In order to make the disturbance more invisible, this paper implements a non-targeted graph-level perturbation. This perturbation method aims to directly replan the edges on the graph instead of directly deleting or reducing edges, so that the disturbance is more invisible. For example, there are three nodes $v1$, $v2$ and $v3$, researchers can remove the connection between $v1$ and $v2$, and add a connection between $v1$ and $v3$. In this case, the number of nodes on the graph is not modified, nor is the number of edges. Therefore, there are multiple ways of perturbation under this module. For Citeseer, Polblogs and Cora datasets, this paper used the Meta-self perturbation method. This perturbation method is two-stage. First, the training model parameters are used to enable the model to learn common parameters in different tasks, and then the fine-tuning stage is used to fine-tune downstream tasks. For the two datasets of Pubmed and Orkut, this paper used the more time-saving A-Meta-Self method for experiments. Specifically, the method designs a meta-self-paced training model consisting of graphical model training patterns to fit the weighting function. This method takes the similarity scores of edge sets as input, and outputs the experimental change perturbation rate. In other words, this paper changes the proportion of the attacking edge from 0% to 25%, and the step size of each increase is 5%. According to the above expression, the experiments in each section of this paper were carried out 20 times, and then the experimental data were analyzed and compared, and the accuracy table of the experiment is reported in Table 2. In the accuracy table, the best data for each individual experimental subsection are indicated by bold annotations in this paper. The following is a full discussion and analysis of the experimental data in the table.

Through the experimental accuracy table in this paper, it can be clearly concluded that under different perturbation rates and multiple rounds of tests on different datasets, the performance indicators of EProGNN are almost the best among all baseline models. Even under some specific scenarios, the performance of EProGNN is extraordinary. For example, in the Cora dataset, when the disturbance rate is 25%, the correct rate of the GCN model has dropped to $48.92\%$, but the correct rate of EProGNN can still maintain a high correct rate of $64.85\%$. In this case, EProGNN is still $1.48\%$ more accurate than the original model ProGNN. The stability of EProGNN is not only reflected in the high perturbation rate, but also shows excellent performance indicators under the low perturbation rate. For example, in the Pubmed dataset, when the perturbation rate is maintained at $5\%$, the rate of decrease in the correct rate of ProGNN has reached $0.74\%$, but the rate of change in the correct rate of EProGNN is only $0.06\%$. Under the same circumstances, the correct decline rates of the GCN and GAT models are $0.07\%$ and $1.14\%$, respectively.

By observing the correct rate table, it can be concluded that in some datasets such as Cora and Citeseer, the rate of decrease in the correct rate of some models is very alarming. Although their model architectures are excellent, such as GCN, GCN-SVD, and GraphSAGE. Because GCN-SVD is specially designed for pointing perturbations, the performance will naturally be greatly reduced for the unspecified attacks used in the experiments in this section. Excellent baseline models such as GraphSAGE under similar circumstances, although their own models have very good quality, but in this specific environment with increasing perturbation rate, their performance is not as good as EProGNN. For example, in the pubmed dataset, under the condition of increasing perturbation rate, the average decrease in the correct rate of EProGNN is only $0.138\%$. Under the same circumstances, the GCN-Jaccard model reached $0.396\%$. The reason for this is that EProGNN uses evolutionary evolution to perform multiple rounds of preprocessing on perturbed graph structure paradigms of different perturbation levels, and can well restore the original canonical graph structure model in complex undifferentiated adversarial noise. Therefore, the EProGNN model under the blessing of the evolutionary algorithm uses a preprocessing learning strategy to defend against adversarial perturbations on irregular graph edge sets, and then realizes multiple rounds of schema updates. The model can effectively remove adversarial edges with different perturbation rates through low-rank and sparse constraints, and then exclude normal edges. It is precisely because of the above reasons that EProGNN can perform better than ProGNN and ProGNN-fs, no matter at a low or high perturbation rate. EProGNN has confirmed through a series of experimental data that the model can fully use evolutionary algorithms and feature smoothing and is reliable and stable in removing large-scale random confrontation disturbance edges.

5.2.2. Against with Targeted Adversarial Perturbation

In this section, this paper used a network perturbation method to verify the performance of EProGNN and its various baseline models from the perspective of multiple datasets. The core idea of Nettack aims to only modify the graph during prediction [48], that is, to let the experimental benchmark model parameters be trained only on the training data. The perturbation occurs after the model is trained or during the experimental verification test phase. Since the experimental benchmark model has been fixed, perturbation cannot affect various parameters or structures of the canonical graph structure model. In the experiments in this section, this paper sets the target perturbation number of each graph structure node in detail, ranging from 0 to 5. In the part of the test set, the principle followed in this paper is that when the degree of the graph node of the normative graph structure exceeds 15, it is defined as the target test node. The correct rate of each benchmark model in the node classification experiment in this link is shown in Figure 2. It can be clearly seen from the above figure that with the increase of the number of disturbances, EProGNN has a high accuracy rate and excellent stability. Whether in the Pubmed dataset or in other datasets such as Cora, the EProGNN model proposed in this paper is basically ahead of other excellent baseline representative models. For example, in the Polbolgs dataset, when the number of perturbed target nodes is 5, the performance of EProGNN is $0.021\%$ higher than that of the GCN-SVD model, reaching an astonishing $96.2\%$. In this case, the performance of EProGNN was even ahead of the original ProGNN model. In addition, the correct rate of EProGNN in this mode is $0.042\%$ higher than that of GAT, GCN, and other models. The demonstration of the experimental line chart fully proves that the model proposed in this paper is successful against targeted adversarial attacks.

Figure 2. Correctness indicators of different baseline models under nettack perturbation mode.

5.2.3. Against with Randomization Adversarial Perturbation

In this section, this paper set the step size to $0.25$, and conducted experiments from $0\%$ to $100\%$ of the random perturbation noise ratio. In this section, the five most representative datasets and multiple baseline models are still set up for accuracy analysis and comparison. All the experimental data results have been shown in Figure 3. From the various experimental data and graphs in the figure, it can be clearly seen that EProGNN has shown excellent resistance to randomized adversarial perturbations under multiple datasets and basically all of them are higher than other baseline models. For example, in the Polblogs dataset, when the disturbance rate is as high as $80\%$, the EProGNN model can still maintain a $90.5\%$ correct rate. In contrast, the correct rate of the classic baseline model GAT is reduced to $84\%$, and the GCN model supported by the SVD framework is also reduced to $87.5\%$. Through the analysis of the experimental results combining three different perturbation modes, it can be said with certainty that EProGNN can fully cope with different complex perturbation modes. Under a variety of perturbation modes, the EProGNN model showed extremely high stability and even surpassed the original model ProGNN for a time. In practical applications, the model is also very practical. The reason is that when faced with different patterns of perturbations, EProGNN can perfectly integrate the evolutionary algorithm and the ProGNN model to resist perturbations.

Figure 3. Correctness indicators of different baseline models under random perturbation mode.

5.3. Effectiveness Analysis of Graph Structure Learning

This model is based on the adversary adding poisoned edges instead of removing normal edges in a clean canonical graph. It is worth noting that the model tends to use evolutionary algorithms combined with traditional methods to learn and strengthen clean canonical graph structures, so as to mitigate the impact of perturbation edges in graphs affected by poisoned edge sets. Therefore, while conducting the above three adversarial experiments, this paper also examined the influence of the weight ratio of adversarial edges and regular edges. Through the analysis of Citeseer and Orkut datasets, this paper finds that the weight of the adversarial edge in the EProGNN model proposed in this paper is much smaller than the edge set of the canonical graph structure during the entire iterative process. In addition, the bilateral weight difference of EProGNN is slightly larger than that of the original model ProGNN. Therefore, this paper can conclude that EProGNN can also successfully suppress the influence of adversarial edges, and then obtain robust and optimal GNN parameters and evolutionary hyperparameters through training.

Through the analysis of the performance in severe perturbation mode, the core idea solved by this model aims to weaken the edge set of the canonical graph structure perturbation by brute force. Specifically, when removing the edge set of the canonical perturbation graph structure, the adjacency matrix of this model will be all zeros. Which means change all the elements in the row and column of the adjacency matrix to 0, but this operation needs to be completed on the original matrix. In this model, the zero-sum matrix is normalized into a unit regular matrix through a specific function, and the purpose is to facilitate edge set prediction. The goal of the edge set prediction module aims to predict whether there is an edge between two nodes of the model. For any of the five datasets, there are node attributes. The positive samples are stored at the edge and endpoints of these datasets. When the prediction task occurs, this model will generate some negative samples, that is, sample some node pairs without edges as negative sample edges, so that the number of positive and negative samples is balanced. Through the experimental data results, it can be observed that when the graph structure is severely impacted, EProGNN can still obtain considerable data by weakening the graph structure. Therefore, EProGNN can also learn rich graph structure information even when the regular graph is seriously impacted randomly or with a target.

5.4. Ablation Experiment

To verify the necessity of the mutation operator and feature smoothing mechanism proposed in EProGNN, ablation experiments were conducted on the Cora and Citeseer datasets. The complete EProGNN model was compared with the following four variants:

• EProGNN w/o RW: The variant removing the Random Walk Mutation.
• EProGNN w/o BT: The variant removing the Betweenness Mutation.
• EProGNN w/o SB: The variant removing the Stochastic Block Mutation.
• EProGNN w/o FS: The variant removing the Feature Smoothing module.

Detailed experimental results are shown in Table 3.

Table 3. Table showing the results of the ablation study of EProGNN on the Cora and Citeseer datasets. The results shown in bold are the best experimental results.

Method	Cora (Acc%)	CiteSeer (Acc%)
EProGNN (Full)	$\mathbf{83}.\mathbf{5}$	$\mathbf{72}.\mathbf{1}$
EProGNN w/o RW	$82.1$	$70.8$
EProGNN w/o BT	$82.4$	$71.0$
EProGNN w/o SB	$81.9$	$70.5$
EProGNN w/o FS	$80.5$	$69.2$

As shown in Table 3, the complete EProGNN model consistently achieves the best performance compared to all variant models. Specifically, it achieves the following:

• Impact of Mutations: Removing any of the mutation operators from the complete model (e.g., not using RW, not using BT, or not using SB) leads to a decrease in performance. This indicates that the different mutation strategies complement each other in exploring the graph structure space and preventing the population from converging prematurely to local optima.
• Impact of feature smoothing: EProGNN’s performance degrades without feature smoothing, indicating that feature smoothing is crucial for filtering high-frequency noise in the feature matrix, especially when the graph structure is perturbed.

In summary, each component plays a unique and indispensable role in enhancing the robustness of EProGNN.

6. Conclusions

The graph neural network series models are extremely vulnerable to cyber perturbations, which can significantly negatively affect the effectiveness of the models. In order to be able to effectively defend against compound types and high complexity adversarial network model perturbations, this paper proposes a new network adversarial perturbation defense method EProGNN. While learning the canonical graph structure paradigm model with high precision and high efficiency and refining GNN training parameters, the model can also effectively analyze and read evolutionary hyperparameters. A large number of extensive experimental data prove that the model proposed in this paper is excellent in resisting perturbations and high robustness, and is obviously better than all other baseline models and original models. Experimental results show that EProGNN achieved an average accuracy improvement of 1.48% compared to the strongest baseline model under high-intensity perturbations (e.g., 25% perturbation rate), validating the effectiveness of the evolutionary strategy in graph structure optimization.

In future work, this paper will further explore more working experiments in the scope of graph structure learning by combining evolutionary algorithms. This paper will explore more graph attribute characteristics to further improve the stability, efficiency and robustness of the graph structure under the given circumstances based on the evolutionary algorithm. Future work of this paper will not only be limited to the GNN series of models, but will also deepen the scope of work to more topology structures and attribute models of node features. In addition, this paper will conduct more detailed research and exploration on the analysis of evolutionary parameters.