Homomorphic Encryption-Based Federated Active Learning on GCNs

Abstract

With the dramatic growth in dataset size, active learning has become one of the effective methods to deal with large-scale unlabeled data. However, most of the existing active learning methods are inefficient due poor target models and lack the ability to utilize the feature similarity between labeled and unlabeled data. Furthermore, data leakage is a serious threat to data privacy. In this paper, considering the features of the data itself, an augmented graph convolutional network is proposed which acts as a sampler for data selection in active learning, avoiding the involvement of the initial poor target model. Then, by applying the proposed GCN as a substitute for the initial poor target model, this paper proposes an active learning model based on augmented GCNs, which is able to select more representative data, enabling the active learning model to achieve better classification performance with limited labeled data. Finally, this paper proposes a homomorphic encryption-based federated active learning model to improve the data utilization and enhance the security of private data. Experiments were conducted on three datasets, Cora, CiteSeer and PubMed, and achieved accuracy rates of 94.47%, 92.86% and 91.51%, respectively, while providing provable security guarantees. Furthermore, the highest malicious user detection accuracy was 88.07%, and the global model test accuracy reached 88.42%, 84.22% and 81.46%, under a model poisoning attack.

1. Introduction

In today’s digital era, graph-structured data are widely used in social networks, transportation networks, and biological networks [1,2,3,4,5]. These graph-structured data usually have complex topologies and contain rich hidden information. However, effective analysis and learning of large-scale graph-structured data usually requires a large amount of labeled data to train machine learning models. Obtaining labeled data is often challenging due to issues such as the scarcity of labeled data and the high cost of labeling. In the field of machine learning, feature selection is one of the key methods to optimize model performance and reduce the risk of overfitting [6,7,8,9,10,11,12,13,14,15]. However, with the high dimensionality of data and complex feature space, traditional feature selection methods lose important feature information, especially in the context of active learning. It is especially critical to select the most informative features to improve model performance.

Graph Convolutional Networks (GCNs) are a specialized class of deep learning models designed for processing graph-structured data. They extend the capabilities of traditional Convolutional Neural Networks (CNNs) to enable direct feature learning and prediction on non-Euclidean spaces (e.g., graphs, social networks, molecular structures) [16,17,18,19,20,21,22,23]. The core concept revolves around local information aggregation to extract feature representations of nodes and their neighborhoods within the graph. In recent years, research on feature selection and active learning based on Graph Convolutional Networks (GCNs) has attracted lots of attention [24,25,26,27]. In 2016, Kipf and Welling first proposed a semi-supervised classification method based on GCNs [12]. The method significantly improved the performance of the semi-supervised learning task by utilizing the relationship information between nodes in graph-structured data. In 2022, Zhang et al. proposed a graph convolutional network method based on node feature convolution [15]. By introducing the node feature convolution mechanism in GCNs, the method captured more nonlinear relationships between node features, improving the modeling capability of graph-structured data. In 2023, Chen et al. proposed a graph convolutional network-based feature selection method specifically for processing high-dimensional low-data-volume data [28]. The method selected the most representative subset of features through the graph structure information transfer mechanism of GCNs, improving the performance and the generalization ability of the model.

Federated Learning (FL) is a distributed machine learning framework that enables multiple clients, such as mobile devices and medical institutions, to collaboratively train a shared model using their local data without directly exchanging raw data [29,30,31]. Its core principle lies in transmitting only model parameters (e.g., gradients) instead of the underlying data, thereby preserving data privacy. Meanwhile, a central server aggregates local updates from participating clients to construct a global model.

In federated learning, how to effectively select data for training local models poses a significant challenge. Furthermore, data leakage and misuse may occur during the processes of data transmission and aggregation [29]. In addition to these privacy and security concerns, the decentralized nature of FL also makes it vulnerable to poisoning attacks. In model poisoning attacks, adversaries manipulate local model parameters and upload carefully crafted malicious updates to the server, aiming to compromise the integrity of the global model [32,33,34,35,36,37,38,39]. Such compromised participants are referred to as malicious users.

Based on the above problem analysis, our technical approach consists of three key modules:

First, the integration of active learning with Graph Convolutional Networks (GCNs) not only facilitates the selection of the most representative and informative samples for labeling, but also allows dynamic adjustment of the data selection strategy according to varying data distributions and task-specific characteristics during the learning process.

Second, homomorphic encryption techniques enable computations on encrypted data without requiring decryption, thereby providing strong guarantees for data security. This ensures that data remain encrypted throughout the entire lifecycle (including transmission, storage, and processing), effectively preventing data leakage and unauthorized use.

Third, a Key Generation Center (KGC) is employed to verify user-submitted updates and detect malicious users. The KGC has the capability to decrypt user ciphertexts. After the server performs model aggregation, the KGC uses the global model parameters to validate each client’s update, identifying and filtering out any potentially malicious contributions. Subsequently, the KGC uploads only the parameters from legitimate users back to the server for further aggregation. This mechanism ensures that the final global model does not incorporate any parameters from malicious users, thereby effectively defending against model poisoning attacks.

The main contributions of this paper are as below.

Firstly, this paper proposes a graph convolutional network model based on feature enhancement. By averaging the feature vectors output from multiple convolutional layers, the multi-level features of nodes can be better captured. The feature similarity between labeled and unlabeled data is utilized so that the unlabeled data that are more different from the labeled data can be selected to obtain more representative data.

Secondly, by applying the proposed GCNs as a substitute for the initial poor target model, this paper proposes an active learning model based on augmented GCNs, which is able to select more representative data, enabling the active learning model to achieve better classification performance with limited labeled data.

Thirdly, this paper proposes a homomorphic encryption-based federated active learning model to improve the data utilization and enhance the security of private data. In the federated active learning model, Client is employed as the proposed active learning based on augmented GCNs, to achieve privacy protection and improve the performance of the active learning model.

2. Preliminaries

2.1. Definition of the Problem

GCNs are deep learning models specialized in processing graph-structured data and incorporating active learning methods to select the most informative nodes for labeling. In GCNs, we let the graph $G$ contain $N$ nodes, denoted as $G=(V,E)$, where $V$ denotes the set of nodes and $E$ denotes the set of edges. Each node i has a feature representation h_i, and these node features can be combined into a feature matrix $X$, where $X={\mathbb{R}}^{N\times D}$, and $D$ denotes the feature dimension of each node. The graph structure can be represented by an adjacency matrix $A$, $A\in {\mathbb{R}}^{N\times N}$, where $A_{ij}$ denotes whether node $i$ to node $j$ is connected by an edge. The node representation of each layer is updated by aggregating the information of its neighboring nodes, denoted as $H^{(l+1)}=f(H^{\left(l\right)},A)$, where $H^{\left(l\right)}$ denotes the node representation of the $l$th layer, f denotes the aggregation function, and $A$ is the adjacency matrix. And in the first layer of GCNs, the input node feature matrix $X$ is usually directly used as the initial node representation matrix $H$, i.e., $H^{\left(0\right)}=X$. By stacking multiple layers, the node representations can be gradually updated for tasks such as node classification, graph classification, etc. The core idea of GCNs is to utilize the connectivity information between the nodes in the graph structure to enhance the capability of representation learning so that it can effectively handle graph data tasks.

In active learning, there is a collection of unlabeled nodes $V_u$ and a collection of labeled nodes $V_l$, where the concatenation of $V_u$ and $V_l$ constitutes the entire collection of nodes, i.e., $V=V_u\cup V_l$. The goal of active learning is to select the most informative nodes for labeling so as to maximize the model performance. In this process, the active learning algorithm selects the most valuable nodes for labeling based on the current model state and the information of the unlabeled nodes.

Combining GCNs and active learning, by training GCNs as samplers for active learning, it is possible to select the most informative nodes for labeling, thus improving the performance of the active learning model.

2.2. Graph Convolutional Network

Graph Convolutional Networks (GCNs) bridge the gap between graph-structured data and convolutional neural networks (CNNs) by adapting the powerful feature extraction capabilities of CNNs to non-Euclidean domains. Graph Convolutional Networks (GCNs) are a specialized class of deep learning models designed for processing graph-structured data. $\mathcal{G}=(\mathcal{V},\mathcal{E})$ denotes an undirected graph, and $\mathcal{E}$ represents the sets of edges that connect different nodes of set $\mathcal{V}$. An adjacency matrix $A$ describes the connectivity between nodes $\mathcal{V}$ in the graph. Typically, $A$ is a sparse matrix, where $A_{ij}$ indicates whether there is an edge or connection between node i and node j. In GCNs, the adjacency matrix $A$ is used to propagate the feature information of the nodes and realize the information interaction between nodes. In GCNs, the relationship between nodes is represented by the adjacency matrix of the graph, and the information propagation and feature extraction of the graph structure are achieved by updating the representation of the nodes by aggregating the information of their neighboring nodes. The feature matrix propagation in GCNs is shown in Equation (1):

$$H^{(l+1)}=\sigma ({\tilde{D}}^{-{\displaystyle \frac{1}{2}}}\tilde{\mathrm{A}}{\tilde{D}}^{-{\displaystyle \frac{1}{2}}}{H}^{(l)}{W}^{(l)})$$

(1)

$H^{\left(l\right)}$ is the node identity matrix of layer $l$, $\tilde{A}=A+I$ is the symmetric adjacency matrix formed by adding the self-connections to the adjacency matrix $A$, $\tilde{D}$ is the degree matrix of $\tilde{A}$, $W^{\left(l\right)}$ is the weight matrix of layer $l$, and $\sigma$ is the activation function, where $H^{(l+1)}=[h_1^{(l+1)},h_2^{(l+1)},\dots ,h_n^{(l+1)}]$.

The propagation rule Equation (1) is derived from spectral graph theory [12], where $\tilde{A}=A+I$ ensures each node aggregates its own features. ${\tilde{D}}^{-{\displaystyle \frac{1}{2}}}\tilde{A}{\tilde{D}}^{-{\displaystyle \frac{1}{2}}}$ symmetrically normalizes the adjacency matrix to avoid numerical instability and maintain scale-invariant features. The weight matrix $W^{\left(l\right)}$ learns layer-specific transformations. This formulation approximates a first-order Chebyshev polynomial filter on the graph Laplacian, balancing local feature aggregation and computational efficiency.

GCNs achieve efficient processing of graph data by defining convolution operations on the nodes of the graph structure. In GCNs, each node exchanges information with its neighboring nodes and updates its own feature representation. Through multi-layered graph convolution operations, GCNs are able to utilize the connectivity between nodes to learn the node representations, thus enabling feature extraction and prediction on graph data.

2.3. Homomorphic Encryption

Homomorphic encryption is a cryptographic method that allows specific types of algebraic operations to be performed on ciphertext, producing an encrypted result that, when decrypted, matches the result of performing the same operations on the corresponding plaintext. In other words, this technology enables operations such as retrieval and comparison to be carried out directly on encrypted data, yielding correct results, without the need to decrypt the data throughout the entire processing process. A homomorphic encryption scheme consists of four probabilistic polynomial time algorithms: public key and key generation algorithm, encryption algorithm, decryption algorithm, and ciphertext evaluation algorithm.

• KeyGen ($1^{\lambda }$): Input security parameter $\lambda$, output public key evk for ciphertext evaluation and secret key sk.
• Enc (pk, m): Input public key pk and message m, output ciphertext c.
• Dec (sk, c): Input ciphertext c and secret key sk, output message m.
• Eval (evk, f, $c_1,\cdots ,c_l$): Input ciphertexts $c_1,\cdots ,c_l$, public key evk for ciphertext evaluation and a function f, where f: ${\{0, 1\}}^l\to \{0, 1\}$. Input ciphertexts $c_1,\cdots ,c_l$ into function f using evk, output ciphertext $c_f$.

2.4. Federated Learning

In federated learning, a central server aggregates locally computed models from clients to train a global model. There are n clients, and each client $C_i$ has its own dataset $S_i$. The central server has initial model $\theta$. After receiving the model $\theta$ from the server, client $C_i$ uses $J_i\left(S_i,\theta \right)={\displaystyle \frac{1}{d_i}}\sum _{(x,y)\in S_i}D(\theta ,(x,y\left)\right)$ to compute the gradient, where $d_i=\left|S_i\right|$ and D is the cost function. Client computes the local model and sends it to the server. Then the server carries out aggregation to derive the global model as follows: $\mathsf{\Theta }={\displaystyle \frac{1}{d}} (d_1{\theta }_1+\dots +d_n{\theta }_n)$. The iterative training process does not terminate until the model converges.

3. Our Methodology

3.1. Feature Enhancement-Based GCNs

The original GCN merely learned the feature information of nodes through the feature matrix. This paper further incorporates the similarity matrix and adjacency matrix based on the feature matrix. The addition of these matrices can enhance the feature learning ability of GCN. Moreover, by using a multi-layer feature fusion method, the ability of GCN to extract features is further strengthened. In this section, we propose a feature enhancement-based graph convolutional network model, which acts as a sampler for data selection, making the selected data more representative. The feature enhancement in the proposed model works in three aspects. Firstly, by combining the feature similarity and adjacency matrix of labeled and unlabeled nodes, the unlabeled nodes with the least correlation with the labeled nodes are selected. Secondly, the representation of nodes is improved by aggregating the node feature information output from different convolutional layers and combining local and global information. Finally, the feature similarity between labeled and unlabeled data is utilized to enable the graph convolutional network to predict whether the input data are labeled or not, so that the unlabeled data which are more different compared to the labeled data can be selected to ensure that the selected data are more representative. The graph convolutional network model based on feature enhancement is shown in Figure 1.

3.1.1. Node Feature Similarity

In GCNs, the adjacency matrix $A$ describes the connectivity between nodes in the graph. Typically, $A$ is a sparse matrix, where $A_{ij}$ indicates whether there is an edge or connection between node i and node j. In GCNs, the adjacency matrix $A$ is used to propagate the feature information of the nodes and realize the information interaction between nodes.

Node features are vectors describing the attributes or characteristics of a node in a graph, which are usually used to represent information on various attributes of a node in the graph. Node feature vectors can provide more key information about the nodes themselves, which can help distinguish the feature differences between different nodes and help the model learn the relationships between nodes in the graph data.

Although the adjacency matrix $A$ can reflect the connection relationship between nodes, when calculating node correlation, not only the adjacency matrix is needed, but also the similarity of node features. The node feature vector reflects the attribute information of the nodes, which can help distinguish the feature differences between different nodes, while the adjacency matrix describes the connectivity relationship between nodes, which can reveal the topology between nodes.

Traditional GCNs rely solely on the adjacency matrix A to propagate node information, which may fail to capture latent relationships between semantically similar but topologically distant nodes. To address this, we introduce a feature similarity matrix S computed via cosine similarity (Equation (2)), as the cosine metric is robust to high-dimensional sparse features. The combined correlation matrix C = αA + βS leverages both topological connectivity and feature semantics. Our ablation study shows that combining A and S improves accuracy by 5.3% compared to using A alone, validating the necessity of the similarity matrix.

We assume that the feature matrix of labeled nodes is $X_l\in {\mathbb{R}}^{N_l\times D}$ and the feature matrix of unlabeled nodes is $X_u\in {\mathbb{R}}^{N_u\times D}$, where $N_l$ is the number of labeled nodes, $N_u$ is the number of unlabeled nodes, and $D$ is the dimensionality of the feature vectors. $S_{ij}$ can be computed by cosine similarity as shown in Equation (2):

$$s_{ij}={\displaystyle \frac{X_{li}\cdot X_{uj}}{\left|\right|X_{li}\left|\right|\cdot \left|\right|X_{uj}\left|\right|}},\forall i\in V_l,\forall j\in V_u$$

(2)

where $X_{li}\in {\mathbb{R}}^D$ denotes the feature vector of labeled node i and $X_{uj}\in {\mathbb{R}}^D$ denotes the feature vector of unlabeled node j. By cosine similarity calculation, we can get a similarity value ranging between [−1, 1], where 1 means completely similar and −1 means not similar at all. The feature similarity matrix $S\in {\mathbb{R}}^{N\times N}$ is shown in Equation (3):

$$S_{ij}=\left\{\begin{array}{l}{s}_{ij}, if i\in V_l,j\in V_u\\ s_{ij}, if i\in V_u,j\in V_l\\ 0, if i,j\in V_l or i,j\in V_u\\ 0, if i=j\end{array}\right.$$

(3)

The adjacency matrix $A\in {\left\{\mathrm{0,1}\right\}}^{N\times N}$ represents the adjacency between nodes, $\tilde{A}=A+I$, where $N=N_l+N_u$ is the total number of nodes. The correlation calculation is shown in Equation (4):

$$C=\alpha \cdot \tilde{A}+\beta \cdot S$$

(4)

where $\alpha$ and $\beta$ are weight parameters to balance the effects of feature similarity and adjacency in the correlation calculation. $S$ is the feature similarity term, which represents the similarity between the feature vector $X_{li}$ of the labeled node i and the feature vector $X_{uj}$ of the unlabeled node j. $\tilde{A}$ is the adjacency term, which represents the adjacency relationship between the labeled node $i$ and the unlabeled node $j$. The graph convolutional network feature matrix propagation in this paper’s method is conducted as shown in Equation (1):

$$H^{(l+1)}=\sigma ({\tilde{D}}^{-{\displaystyle \frac{1}{2}}}C{\tilde{D}}^{-{\displaystyle \frac{1}{2}}}{H}^{(l)}{W}^{(l)})$$

The feature matrix $H^{(l+1)}$ is computed through layer-wise propagation, where the adjacency matrix $\tilde{A}$ is normalized by ${\tilde{D}}^{-{\displaystyle \frac{1}{2}}}\tilde{A}{\tilde{D}}^{-{\displaystyle \frac{1}{2}}}$ to avoid numerical instability. The weight matrix $W^{\left(l\right)}$ is initialized using Xavier initialization, and ReLU is chosen as the activation function due to its effectiveness in mitigating vanishing issues in deep GNNs.

By combining feature similarity and adjacency, the degree of association between labeled and unlabeled nodes is determined through correlation, which in turn, selects the unlabeled nodes that are least relevant to the labeled nodes. This approach mines potential relationships between nodes in the graph and helps to select the unlabeled nodes that have the greatest feature differences from the labeled nodes.

3.1.2. Multi-Layer Feature Fusion

In GCNs, the aggregation of node features is a key process that aims to integrate a node’s own features and the features of its neighboring nodes in order to update the node’s representation. By continuously aggregating information from neighboring nodes, the representation of a node gradually incorporates more information about its surrounding structure and context, thus improving the node feature representation. Nodes throughout the graph go through this aggregation process, allowing each node’s representation to learn global structural information.

In the traditional node feature aggregation process, after multiple information transfers, the node will learn more information about its neighboring nodes, resulting in the loss of key feature information about the node itself. In this paper, we propose a multi-layer feature fusion approach to improve the representation of nodes by aggregating the node feature information output from different convolutional layers and combining local and global information. The feature vector of node $i$ in the $t$th round and $l$th layer is calculated as shown in Equation (5):

$$h_{i,t}^{(l)}=\sigma ({\displaystyle \frac{1}{\sqrt{{\tilde{D}}_{ii}}}}({\displaystyle {\sum }_{j\in N(i)}{h}_{j,t}^{(l-1)}+S_{ij}}){\displaystyle \frac{1}{\sqrt{{\tilde{D}}_{ii}}}}{h}_{i,t}^{(l-1)}{W}_t^{(l-1)})$$

(5)

where $h_{i,t}^{\left(l\right)}$ denotes the eigenvector of node $i$ at round $t$, layer l. $h_{i,t}^{(l-1)}$ and $h_{j,t}^{(l-1)}$ denote the eigenvector of node $i$, node $j$ at round $t$, and layer $l-1$. $W_t^{(l-1)}$ is the weight matrix at round $t$, and layer $l-1$. $N\left(i\right)$ is the set of neighbors of node $i$. $c_{i,j}$ denotes the correlation between node $i$ and node $j$. $\sigma$ is the activation function. The process of multi-layer feature fusion is shown in Equation (6):

$$h_{i,t}={\displaystyle \frac{1}{m}}{\displaystyle {\sum }_{l=1}^m{h}_{i,t}^{(l)}}$$

(6)

where $h_{i,t}$ is the feature vector of node i in the tth round, $h_{i,t}^{\left(l\right)}$ is the feature vector of node $i$ in the $t$th round and $l$th convolutional layer, and $m$ denotes the number of convolutional layers. When all the $h_{i,t}$(i = 1, 2, …, n) are obtained, the feature matrix after the average aggregation at round $t$ can be obtained. The feature matrix after the average aggregation at round $t$ is shown in Equation (7):

$$H_t=[h_{1,t},h_{2,t},\dots ,h_{n,t}]$$

(7)

By averaging the feature vectors of nodes at different convolutional layers, the feature information learned by each node at different convolutional layers is actually being aggregated. This operation enables the model to synthesize multi-layered feature representations from different convolutional layers, combining local and global information and improving the performance of the model for node feature extraction.

The advantage of the multi-layer feature fusion approach is that by combining the feature information learned from different convolutional layers, the model is able to better capture the complex relationships and structural features of the graph data. By integrating local and global information, the multi-layer feature fusion approach can enhance the model’s ability to model node features, thereby improving the model’s performance in the node classification task.

3.2. Active Learning Based on Augmented GCNs

The feature-enhanced graph convolutional network model based on features, instead of the target model, acts as a sampler for data selection, and the active learning can select representative nodes more efficiently, which makes the model pay more attention to those nodes that have an important impact on model parameter updating during the training process, and thus improves the performance of the model. By predicting the nodes based on their feature similarity, representative nodes can be better selected and the efficiency of node selection can be improved. Then, the active learning model is trained based on the graph-structured data selected by this augmented graph convolutional network model, and the active learning model based on an augmented graph convolutional network is proposed, which achieves the maximization of the classification performance of the active learning model on graph-structured data with less labeled data. The active learning model based on an augmented graph convolutional network is shown in Figure 2.

The node classification objective function is shown in Equation (8):

$${\mathcal{l}}_{tra}=-{\displaystyle \frac{1}{\left|V_l\right|}}{\displaystyle \sum _{i\in V_l}{y}_i\log (p_i)}-{\displaystyle \frac{1}{\left|V_u\right|}}{\displaystyle \sum _{j\in V_u}(1-y_j)\log (1-p_j)}$$

(8)

where $y_i$ and $y_j$ are the real labels of node $i$ and node $j$, $V_l$ is the set of labeled nodes, $V_u$ is the set of unlabeled nodes, and $p_i$ and $p_j$ are the prediction results of whether node $i$ and node $j$ are labeled or not. The node feature objective function is shown in Equation (9):

$${\mathcal{l}}_{fea}=KL({\displaystyle \sum _{i\in V_l}{h}^{\prime }(i)\left|\right|}{\displaystyle \sum _{j\in V_u}{h}^{\prime }(j}))$$

(9)

where $h^{\prime }\left(i\right)$ is the aggregated feature vector of labeled node $i$, and $h^{\prime }\left(j\right)$ is the aggregated feature vector of unlabeled node $j$. By combining Equations (8) and (9), the complete objective function for training the graph convolutional network model can be obtained as shown in Equation (10):

$${\mathcal{l}}_{GCN}={\lambda }_1{\mathcal{l}}_{tra}+{\lambda }_2{\mathcal{l}}_{fea}$$

(10)

where ${\lambda }_1$ and ${\lambda }_2$ are hyperparameters that determine whether the graph convolutional network effectively distinguishes between nodes with and without labels.

In this approach, the classification layer of the graph convolutional network is binary, where nodes with features similar to those of the labeled data are likely to be predicted as labeled nodes, while the opposite case will be predicted as unlabeled nodes. The cost of labeling nodes is reduced by selecting nodes that need to be manually labeled through the confidence level of the graph convolutional network model on the nodes. Based on the predicted probability to determine whether a node has labeled data or not, the $b$ nodes with the smallest probability value of being predicted as unlabeled nodes are selected to be manually labeled and the labeled nodes are added to the set of labeled nodes $V_l$.

For the classification task, the active learning model is trained using the set of labeled nodes $V_l$. The loss function is shown in Equation (11):

$$J(\theta )=-{\displaystyle \frac{1}{N}}{\displaystyle {\sum }_{n=1}^N{\displaystyle {\sum }_{i=1}^C{y}_{n,i}log({\widehat{y}}_{n,i})}}$$

(11)

where $N$ is the number of samples in the training set; $C$ is the number of categories; $\theta$ denotes the parameters of the active learning model; $y_{n,i}$ is the true label value of the $i$th category of the $n$th node; and ${\widehat{y}}_{n,i}$ is the prediction probability of the model for the $i$th category of the $n$th sample. By minimizing the cross-entropy loss function, the prediction result of the active learning model can be made as close as possible to the true label, thus improving the classification accuracy of the active learning model.

3.3. Homomorphic Encryption Algorithm Based on Identity Authentication

To protect the client’s private data from leakage, we introduce homomorphic encryption to resist against model poisoning attacks in federated learning. The legitimacy of the client needs to be verified. In this paper, we propose a homomorphic encryption algorithm based on authentication using the Paillier algorithm. The algorithm can verify the functionality of client legitimacy, while KGC is a trusted third party. Identity authentication acts to verify whether participants in federated learning are legitimate users or malicious adversaries. By calculating the mean, median and standard deviation of the first-round gradient data, abnormal data can be identified, and then the source of malicious users can be identified. The identified malicious users will be isolated in the following submitting stages, achieving the purpose of identity authentication. Eventually, a homomorphic encryption algorithm based on identity authentication is implemented. In the proposed homomorphic encryption algorithm, only data from honest users are aggregated and decrypted, which ensures the isolating of malicious data, preserving the client’s private data from leakage. The proposed homomorphic encryption scheme consists of five core algorithms: Key Generation, Encryption, Verification, Aggregation, and Decryption. The homomorphic encryption algorithm based on authentication is shown in Figure 3. The detailed homomorphic encryption algorithm is as follows.

Key generation: KGC selects two large prime numbers $p$ and $q$ and calculates $n=p*q$; then it calculates $\lambda =lcm\left(p-1,q-1\right)$, where $lcm$ denotes the least common multiple; it then randomly selects an integer $g$, $g\in Z_{n^2}^{*}$, and calculates $\mu =L(g^{\lambda }mod n^2{)}^{-1}mod n$, where $L\left(x\right)=(x-1)/n$. The master key pair for generating the KGC can be represented as follows: master public key $mpk=(n,g)$, master private key $msk=(\lambda ,\mu )$. For each client $i$, a large integer $z_i$ is randomly generated and ensures that $z_i$ is mutually prime with $n$. The client public key $\left({pk}_i\right)$ is obtained by performing a computation on the client private key (${sk}_i)$. The key pair for each client can be expressed as follows: private key ${sk}_i=z_i$, public key ${pk}_i=g^{z_i}mod n^2$.

Encryption: Each client uses the master public key $mpk$ to encrypt its locally trained model parameters to obtain the ciphertext model parameter $c_i$. The encryption process can be expressed as follows: $c_i=Enc\left({\theta }_i,mpk\right)=g^{{\theta }_i}*{r_i}^{n}mod n^2$, where $r_i$ is a random number.

Verification: KGC uploads the first round of received $c_i$ directly to the server, which obtains the encrypted global model parameter $c$ by the homomorphic encryption algorithm and sends it down to KGC. KGC decrypts the global model parameter $\theta$ of the first round by using the master private key $msk$, and the decryption process can be expressed as follows: $\theta =Dec\left(c,msk\right)=L\left(c^{\lambda }mod N^2\right)*\mu mod n$. KGC decrypts using the master private key $msk$ to obtain the model parameter ${\theta }_i$ for each client, and the decryption process can be expressed as follows: ${\theta }_i=Dec\left(c_i,msk\right)=L\left(c_i^{\lambda }mod N^2\right)*\mu mod n$. It then calculates the cosine similarity ${cos}_i=\theta ·{\theta }_i/\left|\right|\theta \left|\right|*\left|\right|{\theta }_i\left|\right|$ of the model parameters to the global model parameters for all clients. The mean ($\widehat{\mu }s$), median ($\overline{\mu }s$) and standard deviation ($\sigma s$) of the cosine similarity of all legitimate clients are then calculated. If $\widehat{\mu }s<\overline{\mu }s$, the cosine similarity value ${cos}_i$ will be lower than $\overline{\mu }s$ for most potentially illegitimate clients. If ${cos}_i<\overline{\mu }s-\xi \sigma s$, client $i$ is illegitimate, and $\xi$ is a hyper-parameter.

Aggregation: Starting from the second iteration loop, KGC removes the model parameters uploaded by illegitimate clients and uploads the remaining ciphertext model parameters to the server, which performs aggregation by homomorphic encryption and sends the aggregated ciphertext global model parameters to KGC.

Decryption: The KGC decrypts using the master private key $msk$ to obtain the global model parameters. Then it encrypts and sends it down using the legitimate client’s public key ${pk}_i$, and client $i$ decrypts it using its own private key ${sk}_i$ to obtain the plaintext global model parameters to update the local model.

3.4. Homomorphic Encryption-Based Federated Active Learning

Federated learning allows multiple participants to train models at the same time, which solves the problem of data sharing but also carries the risk of privacy breaches. Homomorphic encryption allows computations to be performed on encrypted data without the need to decrypt the data. This means that in federated learning, participants can share encrypted data or model parameters without exposing the original data, thus realizing privacy protection. Combining homomorphic encryption algorithms with federated learning can further enhance data privacy and security.

In this section, active learning privacy protection is realized for graph-structured data, and a federated active learning system based on homomorphic encryption is proposed by designing a homomorphic encryption algorithm based on authentication, as shown in Figure 4. When handling graph-structured data, the system employs an active learning model based on augmented graph convolutional networks to enhance training efficiency and model performance while ensuring data privacy.

Specifically, for the graph-structured data, federated active learning is implemented on datasets, Cora, CiteSeer, and PubMed, respectively. The specific process includes the active learning training phase, encryption phase, authentication phase, federation aggregation phase, and decryption phase.

Active Learning Training Phase: There are $M$ clients, and each client $i$ trains an active learning model by a feature-enhanced graph convolutional network using the structural dataset of this graph-structured data to obtain the local model parameters ${\theta }_i$.

Encryption phase: client $i$ encrypts the local model parameters ${\theta }_i^t$ of the t th round of their respective training using the master public key $mpk$ to obtain $E_{mpk}\left({\theta }_i^t\right)$, respectively, and uploads the encrypted local model parameters to KGC.

Authentication phase: The KGC verifies the legitimacy of client $i$ according to the homomorphic encryption algorithm based on authentication and uploads the cipher model parameters of the legitimate client to the server.

Federated Aggregation Phase: The server takes the cipher model parameters uploaded by KGC received in the $t$th round and aggregates them against the cipher model parameters of legitimate client $i^{\prime }$.

$$Aggregate{d}_{model}\left(w_i,E_{pub}({\theta }_i^t)\right)={\displaystyle {\sum }_{i=1}^M{w}_i*E_{pub}({\theta }_i^t)}$$

(12)

Decryption phase: The server sends the aggregated encrypted model parameters to KGC, which decrypts the global model parameter ${\theta }^t$ using the master private key $msk$. KGC encrypts the encrypted global model parameters using the public key ${pk}_{i^{\prime }}$ of the legitimate client $i^{\prime }$ and sends it to the legitimate client $i^{\prime }$, which decrypts the encrypted global model parameters using the private key ${sk}_{i^{\prime }}$ and obtains the global model parameters ${\theta }^t$ in plaintext.

Active Learning Model Parameter Update Phase: Legitimate clients $i^{\prime }$ use the decrypted global model parameters to update the local model parameters and improve the local active learning model performance.

Since client data are encrypted using the homomorphic algorithm during the federated learning process, the ciphertext remains encrypted both during transmission and while stored on the server. This prevents the leakage of information related to gradients. Before the ciphertext is submitted to the server for computation, the key generation center performs authentication, which can identify non-compliant data and malicious users. As a result, this mechanism effectively defends against poisoning attacks.

4. Experiments

4.1. Experimental Setup

4.1.1. Datasets

As shown in

Table 1. Statistics of three datasets.

Dataset	Nodes	Edges	Features	Classes
Cora	2708	5429	1433	7
CiteSeer	3327	4732	3703	6
PubMed	19,717	44,338	500	3

, Cora, Citeseer and PubMed are three commonly used academic citation network datasets widely used in the field of graph neural network research [8,9,12,15,19]. The Cora dataset consists of 2708 papers in the field of machine learning, and each paper is represented by an eigenvector containing word-frequency statistics, and the edges indicate the citation relationships between papers. It is mainly used for the node classification task, by categorizing the papers into seven categories according to their research areas, namely, “Case_Based”, “Genetic_Algorithms”, “Neural_Networks”, “Probabilistic_Methods”, “Reinforcement_ Learning”, “Rule_Learning”, and “Theory”. Edges denote citation relationships between papers, i.e., if a paper cites another paper, then there is an edge between the corresponding nodes. The Citeseer dataset contains 3327 academic papers and also covers citation relationships between papers. Each paper is represented by a feature vector with dimension 3703. Citeseer is also widely used for node classification tasks and is categorized into six categories based on different research areas in computer science, namely, “CS.CC”, “CS.DS”, “CS.GL”, “CS.IR”, “CS.LG”, and “CS.NE”. Similar to the Cora dataset, edges indicate citation relationships between papers. The PubMed dataset consists of 19,717 biomedical articles, each with a feature vector of dimension 500, with a total of three categories: “Cardiovascular_Diseases”, “Cancer_Research”, and “Neurology”. The edges represent citation relationships between documents, which are mainly used for node classification and document relationship mining tasks. Researchers can utilize PubMed datasets to study literature relationships in the biomedical field, infer correlations between literature and classify topics, and provide useful references and insights for medical research. The characteristics and size of these three datasets make them important benchmark datasets for studying graph neural network models and algorithms.

4.1.2. Baselines

Random: Random method is a simple baseline method for decision making or experimentation by random selection.

Entropy-based method [11]: Information entropy-based methods utilize the concept of information entropy for decision making or selection and are commonly used for feature selection and decision tree construction.

Coreset [10]: Coreset method is a data sampling technique that preserves the main features of the original dataset by selecting representative samples.

Degree: Degree methods are commonly used in graph theory for iterative labeling of nodes based on their degree and are used to measure the importance of nodes.

AGE [8]: AGE is a method used to learn causal relationships in graphs, inferring causal relationships between variables through greedy algorithms and equivalence classes.

ANRMAB [9]: ANRMAB is a multi-armed gaming machine algorithm for decision making problems in non-smooth environments by dynamically adapting to changes in the environment.

4.1.3. Assessment Indicators

In machine learning and classification problems, there is a close relationship between accuracy, precision, recall, specificity and F1 score, which are important metrics for evaluating the performance of a classifier. Accuracy is the ratio of the total number of samples correctly predicted by the classifier to the total number of samples, and is a measure of the overall performance of the classifier. Precision rate is the ratio of the number of samples predicted by the classifier as positive categories and correct to the number of samples predicted as positive categories, which emphasizes the accuracy of the classifier in predicting positive categories. Recall is the ratio of the number of samples correctly predicted as positive categories by the classifier to the number of samples that are actually positive categories, which evaluates the ability of the classifier to recognize positive categories. The F1 score combines the precision and recall and is the reconciled average of the precision and recall, which is used to evaluate the performance of the classifier in a comprehensive manner. Sensitivity is the proportion of samples that are actually positive categories that are correctly recognized as positive categories by the classifier, also known as true instance rate or check all rate. Specificity is the proportion of samples that are actually negative categories that are correctly recognized as negative categories by the classifier, and evaluates the ability of the classifier to recognize on negative categories.

4.2. Results and Analysis

Based on the experimental results, it can be seen that there is a difference in the performance of the model on the three different datasets. As shown in

Table 2. Classification performance on three datasets.

Dataset	Acc (%)	Pre (%)	Recall (%)	Spe (%)	F1 Score (%)
Cora	94.47	93.99	94.29	99.08	94.05
CiteSeer	92.86	92.64	91.29	98.56	91.78
PubMed	91.51	91.90	90.79	95.48	91.26

, the best results are achieved with the Cora dataset in terms of accuracy, precision, recall, specificity and F1 score, showing its superiority on the classification task. This may be due to the fact that the Cora dataset has a clearer distribution of categories and higher quality data, which makes it easier for the model to learn and generalize to new data. The slightly lower recall and F1 scores of CiteSeer may be affected by the imbalance of categories in the dataset, which results in the model’s poor performance on a few categories. The PubMed dataset, on the other hand, has relatively low specificity and accuracy, possibly due to poor data quality or feature selection.

These results suggest the importance of data characteristics that need to be considered when designing models for different datasets. In practical applications, careful analysis and preprocessing of datasets, and selection of appropriate feature engineering methods, as well as model tuning strategies, can help improve model performance and generalization. Comprehensive consideration of data quality, feature selection, model complexity, and training strategies can be targeted to improve model performance on different datasets and increase the accuracy of the classification task.

4.2.1. Classification Experiment

Based on the analysis of the experimental results, it can be observed that on the Cora, CiteSeer and PubMed datasets, this paper’s method is able to achieve a high level of accuracy when only 40% labeled data are used. As shown in Figure 5, as the amount of labeled data increases from 10% to 40%, the accuracy of the model on each dataset gradually improves and finally reaches a high level in the case of 40% labeled data. The accuracy increased from 48.15% to 94.47% on the Cora dataset, from 55.22% to 92.48% on the CiteSeer dataset, and from 53.16% to 91.51% on the PubMed dataset. This indicates that the model is able to select more representative data on these datasets, thus achieving higher classification accuracy. This suggests that the data selection by the method in this paper helps the model learn more representative features and improves generalization to unlabeled data. This approach not only improves the accuracy, but also highlights the model’s generalization ability and learning effect on different datasets, which provides an effective solution to address the situation where labeled data are scarce or costly to label.

4.2.2. Ablation Experiment

When performing ablation experiments on Cora, CiteSeer and PubMed datasets with different numbers of GCN convolutional layers, some clear trends were found. As shown in Figure 6, on the Cora dataset, as the number of GCN convolutional layers increases, the F1 score is highest at GCN 3 convolutional layers, with the highest value of 94.05%, showing that increasing the number of convolutional layers helps the model to better understand the complex relationships between the data. Similarly, the highest F1 score of 91.78% was achieved with the GCN 3-layer convolutional layer on the CiteSeer dataset, with the three-layer convolutional layer performing better for the model compared to fewer or more convolutional layers. The results on the PubMed dataset show a similar trend, with the highest F1 score of 91.26% for the GCN three-layer convolutional layer, showing that increasing the number of convolutional layers to three helps to improve the model’s classification performance. Comprehensively analyzing these experimental results, it can be concluded that choosing a three-layer GCN convolutional layer may be a more appropriate choice on the Cora, CiteSeer, and PubMed datasets. The three-layer GCN convolutional layer shows better performance on these datasets, and is able to achieve higher F1 scores on different datasets. This suggests that increasing the number of convolutional layers to three in these cases helps the model to learn the complexity of the data better, improves the classification performance of the model, and exhibits better generalization ability.

4.2.3. Comparison Experiment

According to the experimental results, the F1 scores of seven different methods on Cora, CiteSeer and PubMed datasets are analyzed. As shown in Figure 7, on the Cora dataset, it is observed that the F1 scores of this paper’s methods gradually increase from 10.33% to 94.05% under different labeled data ratios, showing a significant growth trend. For the CiteSeer dataset, it is observed that the performance of the different methods also improves with the increase in the labeled data ratio. Specifically, the F1 score of this paper’s method on this dataset increased from 15.56% to 91.78%, showing a high growth rate. On the PubMed dataset, the F1 scores of the different methods similarly show an increasing trend with the increase in the proportion of labeled data. The performance of the method in this paper is also highlighted again by the F1 scores, which increase from 30.18% to 91.26%, demonstrating a clear advantage. Other methods such as AGE, ANRMAB, Degree, Entropy-based method, Coreset, and Random also show an increasing trend with the increase in the proportion of labeled data, but among these methods, the method in this paper has the highest performance on the Cora, CiteSeer, and PubMed datasets. Combining the experimental results of the three datasets, this paper’s method performs well on Cora, CiteSeer and PubMed datasets, and relative to the other six methods, this paper’s method achieves higher F1 scores with different labeled data proportions. Therefore, based on the presented data, it can be concluded that the method proposed in this paper performs best under these experimental conditions and has better generalization performance and stability.

4.2.4. Precision, Recall, Specificity Analysis

Based on the experimental results, we can analyze the performance of the method in this paper using three evaluation metrics: precision, recall and specificity. The datasets include Cora, CiteSeer and PubMed. As shown in Figure 8, we can see that for the precision (precision) on the three datasets, the performance of this paper’s method is 93.99% (Cora), 92.64% (CiteSeer) and 91.9% (PubMed). Precision measures the proportion of all samples classified as positive classes in which the model is truly positive, so high precision indicates that the model is highly accurate in classifying positive classes. Based on the data, this paper’s method performs best on the Cora dataset, while also maintaining a high level of accuracy on the CiteSeer and PubMed datasets. Recall evaluates the ability of the classifier to recognize positive categories. The recall performance of this paper’s method on the three datasets is 94.29% (Cora), 91.29% (CiteSeer) and 90.79% (PubMed). Sensitivity measures the model’s ability to correctly identify positive class samples, i.e., the proportion of all true positive class samples that are correctly identified as positive. The data show that this paper’s method has the highest sensitivity on the Cora dataset, while it also performs well on the CiteSeer and PubMed datasets. The experimental results show that the specificity (specificity) performance of this paper’s method on the three datasets is 99.08% (Cora), 98.56% (CiteSeer), and 95.48% (PubMed), respectively. The specificity metric measures the model’s ability to correctly identify negative class samples, i.e., the proportion of all true negative class samples that are correctly identified as negative. According to the data, this paper’s method has the highest specificity on the Cora dataset, while it also has a high level of specificity on the CiteSeer and PubMed datasets. In summary, this paper’s method performs well on the Cora, CiteSeer and PubMed datasets in the three evaluation metrics of accuracy, sensitivity and specificity. Specifically, on the Cora dataset, the method performs best in terms of accuracy, sensitivity and specificity; while on the CiteSeer and PubMed datasets, it maintains a high level of overall performance, albeit with slight differences.

4.2.5. Malicious User Detection Rate

According to the experimental results, the method proposed in this subsection shows a high malicious user detection accuracy in terms of using graph-structured data when 10 clients are involved and each of them contains 5 malicious clients. As shown in Figure 9, during the federated learning process, a significant improvement in malicious user detection accuracy is achieved as the number of training sessions increases. When using graph-structured datasets, the highest malicious user detection accuracy is 88.07% for the Cora dataset, 83.9% for the CiteSeer dataset, and 81.74% for the PubMed dataset. Similarly, the malicious user detection accuracy when dealing with graph-structured data shows a significant growth trend as the number of trainings increases. These experimental results demonstrate that the method proposed in this subsection exhibits high accuracy in detecting malicious users, achieving satisfactory results when graph-structured data are used. This further validates the effectiveness and practicality of the homomorphic encryption algorithm based on authentication proposed in this subsection, which provides a reliable and efficient solution for malicious user detection in federated learning systems.

4.2.6. Global Model Performance Analysis

Based on the analysis of the experimental results, the method proposed in this subsection performs well in processing graph-structured data (e.g., Cora, CiteSeer, and PubMed). In the case of malicious clients using model poisoning attacks, the accuracy of the global model test is significantly improved, demonstrating the ability of the method in this subsection against malicious attacks. As shown in Figure 10, on the graph-structured data, the global model test accuracy for Cora, CiteSeer and PubMed datasets shows a significant increase from 29.06%, 27.03% and 24.86% to 88.42%, 84.22% and 81.46%, respectively. The effectiveness of the homomorphic encryption algorithm based on authentication is verified, with the algorithm successfully detecting and defending against attacks from malicious clients and ensuring the stable performance of the global model. When dealing with graph-structured data, the methods proposed in this subsection perform well, showing a wide range of applicability and robustness. Despite five malicious clients using the model poisoning attack, the test accuracy of the global model is still significantly improved, confirming that the method proposed in this subsection can effectively filter the interference of malicious clients and guarantee the security and performance of the federated learning system. In summary, the method proposed in this subsection achieves satisfactory results in processing graph-structured data, effectively counteracts attacks from malicious clients, and ensures the security and performance of the federated learning system.

5. Conclusions

Considering the features of the data itself, an augmented graph convolutional network is proposed and acts as a sampler for data selection in active learning, avoiding the involvement of the initial poor target model. The proposed GCNs not only fully utilize the feature similarity between labeled and unlabeled data, but also solve the problem of unrepresentative selected data, significantly improving the model performance in active learning. By applying the active learning model based on enhancement graph convolutional network into federated learning, this paper proposes a homomorphic encryption-based federated active learning. Experimental results on multiple datasets show that the method performs well in terms of accuracy, sensitivity and specificity, especially in the node classification task, to achieve significant performance improvement. Our experiments reveal that the computation of the feature matrix significantly impacts model performance. Proper normalization (e.g., symmetric normalization) not only stabilizes training but also enhances the model’s ability to capture long-range dependencies in the graph. The proposed homomorphic encryption-based federal active learning not only guarantees the data security, but also improves the data utilization and the classification performance of the active learning model. In future work, we will extend the proposed model to image data to widen its application prospects.