Robust Hierarchical Federated Learning with Anomaly Detection in Cloud-Edge-End Cooperation Networks

Abstract

Federated learning (FL) enables devices to collaborate on machine learning (ML) model training with distributed data while preserving privacy. However, the traditional FL is inefficient and costly in cloud–edge–end cooperation networks since the adopted classical client-server communication framework fails to consider the real network structure. Moreover, malicious attackers and malfunctioning clients may be implied in all participators to exert adverse impacts as abnormal behaviours on the FL process. To address the above challenges, we leverage cloud–edge–end cooperation to propose a robust hierarchical federated learning (R-HFL) framework to enhance inherent system resistance to abnormal behaviours while improving communication efficiency in practical networks and keeping the advantages of the traditional FL. Specifically, we introduce a hierarchical cloud–edge–end collaboration-based FL framework to reduce communication costs. For the framework, we design a detection mechanism as partial cosine similarity (PCS) to filter adverse clients to improve performance, where the proposed lightweight technique has high computation parallelization. Besides, we theoretically discuss the influence of the proposed PCS on the convergence and stabilization of FL. Finally, the experimental results show that the proposed R-HFL always outperforms baselines in general cases under malicious attacks, which further shows the effectiveness of our scheme.

1. Introduction

With the rapid development of communication technologies, a large amount of heterogeneous data is generated at network edges [1,2]. To enable intelligent network services, such as vehicle scheduling [3,4], image processing [5], edge computing and caching [6,7], machine learning (ML), a data-driven technology, can be applied. Given that the centralized learning paradigm of cloud computing generates huge transmission delays and risks user privacy, it is required to use an efficient and privacy-preserving paradigm to online learn policies for real-time intelligent inference in heterogeneous cloud–edge–end cooperation networks.

As one of the privacy-preserving distributed learning paradigms, federated learning (FL) [8,9] can train a common ML model collaboratively from heterogeneous data generated at network edges. In traditional FL (or FedAvg [8]), the client-server framework is generally adopted: the ML model is downloaded to each client, locally updated in parallel, and uploaded to the server for parameter averaging. The framework includes two popular architectures in practical deployments: cloud-based FL [10,11,12] and edge-based FL [13,14,15]. The cloud-based FL sets the cloud as the server, which permits a large number of clients to participate in the FL training for advantageous global performance, but also incurs high communication overhead and time delays due to network congestion and large model size. The edge-based FL sets the edge as the server, which only holds partial clients near the edge server in the learning system for efficient training, which yields a biased approximation of the global optimum with poor performance. Thus, using the client-server framework without considering cloud–edge–end cooperation in networks leads to the traditional FL not being suitable for general network applications requiring high performance with low latency [3,4,5,6,7].

To combine the advantages of the two FL types and leverage the real-world network communication system for intelligent network applications, hierarchical federated learning (HFL) [16] is proposed as a bi-level cloud–edge–client framework that cooperatively trains ML models on the cloud and edge sides. Specifically, on the edge side, multiple edge-based FL updates are performed in parallel: each edge server communicates with disjoint clients to derive the respective edge model. Then, on the cloud side, a centralized cloud server communicates with edge servers to average all uploaded edge models (which can regard edge servers as clients in the cloud-based FL). The advanced paradigm technically solves the trade-off between communication and computation while inheriting the privacy preservation characteristics from the traditional FL. However, this inherited property (privacy preservation) also leads to the HFL servers having no access to raw data or full control over participators’ behaviours. Consequently, malicious or malfunctioning heterogeneous clients may upload incorrect or noisy model values to the server (originally named Byzantine attacks or poisoning), which strongly hurts the FL training and degrades the final performance [17].

To avoid abnormal behaviours degrading training performance, we design a new HFL method with a lightweight detection mechanism that can distributively detect adverse clients’ uploaded information at the edge side for system robustness. In particular, we divide and offload the detection task to the edge, i.e., deploying a partial detector on each edge server for identification. In addition, these partial detectors are simply established by applying the cosine similarity technique, which detects abnormal behaviour by measuring the distance between the previous edge aggregation model and the uploaded client model.

The main contributions are summarized in this paper as follows:

• A novel method called R-HFL is proposed to ensure the training performance under Byzantine attacks by the distributed anomaly detection mechanism customized for the HFL framework.
• The effectiveness and convergence of our proposed method are mathematically discussed in the general non-convex case.
• Numerical results are provided to experimentally show that our proposed algorithm can effectively minimize the negative impact of abnormal behaviours, which further illustrates the feasibility of distributive detection in the HFL system.

The remainder of the paper is organized as follows. In Section 2, related works are introduced. In Section 3, the learning system architectures are described. In Section 4, the new R-HFL algorithm is proposed and discussed. In Section 5, the effectiveness and convergence of R-HFL are analysed. In Section 6, experiment results are presented. Finally, the paper is concluded in Section 7.

2. Related Work

Federated learning includes horizontal federated learning, vertical federated learning and federated transfer learning, where we focus on the first type in this paper and call it federated learning (FL) for short. FL was first proposed in [8], named FedAvg, which provided a guideline on local parameter aggregation. Thus, many works followed FedAvg to analyse the theoretical performance. Refs. [18,19,20,21] analysed the FL convergence performance in iid settings (i.e., raw data is independent-and-identically-distributed), where Hao and et al. in [18] showed the result for non-convex optimization objectives. Additionally, as heterogeneity is ubiquitous in a realistic environment, some researches tried to analyse the FL convergence on non-iid (non-independent-and-identically-distributed) data in inconsistent computation capability. In the works [22,23,24], the FL convergence was analysed in terms of the error bound under the constraint of the difference between the local and global gradients. In particular, the results in [25] pointed out that FedAvg was not appropriate to train non-i.i.d data. In [26], the authors proved that when local training steps were heterogeneous among local clients, many federated learning schemes including FedAvg would converge to non-optimal stationary model parameters. Other works are devoted to deploying FedAvg to practical networks. Since the upload of poorly trained local models (due to statistical and system heterogeneity) occupies the network bandwidth, which increases communication overhead and influences the learning system efficiency, client scheduling is a critical issue [10,27]. Those works discussed scheduling policies that sample a subset of clients in aggregation for communication cost-saving, while [28] studied the problem in wireless communication scenarios, where communication link quality was optimized in the parameter aggregation stage of FL. Besides client scheduling, hierarchical FL [16] is another promising approach for effectively reducing communication overhead, which improves the system training efficiency by considering the practical network architecture. Works as [29,30] aimed to optimize the HFL performance via solving resource allocation problems as client-edge assignments.

The above works focused on performance improvement for FL in data and system aspects, which lacked discussion on learning system security while the distributed learning framework across multiple clients left FL vulnerable. Recently, some defensive techniques [31,32,33,34,35] have been proposed against poisoning attacks. For instance, [31] discussed methods of abnormal client update identification in FL. The work [32] raised the Krum model to check a deviation in input parameters from each participator. The work [33] designed the Auror scheme, which can detect malicious users and generates corresponding accurate models by inserting a clustering operation on each local update before an aggregation round as a defensive scheme. The works [34,35] proposed auto-encoder-based approaches to help identify malicious clients. Whereas, these studies aimed to build defensive mechanisms for the classical distributed ML or the traditional FL, i.e., they deployed a single detector for the server in the classical two-tier client-server architecture, which cannot be directly applicable to the HFL system since the layered structure prevents the above algorithms from accessing all essential information uploaded by clients. Additionally, some of them as [34,35] have heavy extra communication and computation costs to train a neural network for assisting anomaly detection. Thus, it is required to design a new detection mechanism adopted for the HFL system in this paper.

3. Learning System Architecture

In this section, we first review the traditional FL with the client–server architecture based on the typical algorithm FedAvg. Then we introduce the hierarchical FL framework with the advantageous cloud–edge–client architecture.

3.1. Traditional Federated Learning

For all participating clients in $\mathcal{N}$, the FL optimization objective is usually depicted as follows:

$$\underset{\mathit{x}\in {\mathbb{R}}^d}{min}f\left(\mathit{x}\right):=\frac{1}{\left|\mathcal{N}\right|}{\sum }_{i\in \mathcal{N}}{f}_i\left(\mathit{x}\right)$$

(1)

where d denotes the dimension of the model parameterized vector $\mathit{x}$, and $f_i\left(\mathit{x}\right)$ represents the sub-objective function defined by the local ML model on each client i with $f_i\left(\mathit{x}\right):={\mathbb{E}}_{{\mathbf{\vartheta }}_i\sim D_i}\left[f(\mathit{x};{\mathbf{\vartheta }}_i)\right]$. Additionally, $D_i$ is the local dataset of client i while ${\mathbf{\vartheta }}_i$ is a data sample from it. We depict a round of the FedAvg algorithm as three stages: (i) (server) broadcasting, (ii) (client) updating, and (iii) (server) aggregation. To be specific, in the k-th communication round, we have the following.

(i) Broadcasting: The server initializes (if necessary) and then broadcasts a common global model ${\mathit{x}}^k$ to each participating client $i\in \mathcal{N}$.

(ii) Updating: Each client i initializes the local model ${\mathit{x}}_{i,0}^k$ as ${\mathit{x}}^k$ and sequentially updates the model by performing stochastic gradient descent (SGD) with a step-size $\eta$ on $f_i$,

$${\mathit{x}}_{i,j+1}^k\leftarrow {\mathit{x}}_{i,j}^k-\eta g_i\left({\mathit{x}}_{i,j}^k\right),j=0,1,...,\tau -1,$$

(2)

where ${\mathit{x}}_{i,j}^k$ denotes the updated local model in the j-th step SGD, $g_i(·)$ represents the stochastic gradient of $f_i(·)$ (i.e., the short of $\nabla f(·;{\mathbf{\vartheta }}_i)$ with ${\mathbf{\vartheta }}_i\sim D_i$). Besides, while the local update step $j\to \tau$, client i uploads its trained model to the server.

(iii) Aggregation: The server receives aggregates all trained local models to generate a new global one for the next training round, i.e.,

$$\begin{array}{cc}\hfill & {\mathit{x}}^{k+1}\leftarrow \frac{1}{\left|\mathcal{N}\right|}{\sum }_{i\in \mathcal{N}}{\mathit{x}}_{i,\tau }^k.\hfill \end{array}$$

(3)

The whole process is completed when $k\to K$.

We summarize the process as Algorithm 1 to search for the optimal solution of the objective (1), which equals the procedure of cloud-based FL when regarding the cloud as a server. In addition, in the case of the edge-based FL, only a fraction of clients near the edge server (i.e., ${\mathcal{N}}_m\subset \mathcal{N}$, $\forall m\in \mathcal{M}$) can participate in the training process since the server has a limited communication range. In other words, in Line 3 of Algorithm 1, the subset ${\mathcal{N}}_m$ with any edge server m is input into the ServerUpdate operation in place of the whole client set $\mathcal{N}$, where $\mathbf{ServerUpdate}(·;·)$ ensembles all operations presented in the preceding subsection (i.e., broadcasting, updating and aggregation).

Algorithm 1 Traditional Client-Server FL.

Input: local update step $\tau$, total communication round K Output: optimized global model ${\mathit{x}}^{*}$

1: server randomly initializes global model ${\mathit{x}}^k$ with counter $k=0$ 2: for server in each round $k=0,1,\cdots ,K-1$ do 3:       ${\mathit{x}}^{k+1}\leftarrow \mathbf{ServerUpdate}({\mathit{x}}^k;\mathcal{N})$ 4: end for 5: server returns ${\mathit{x}}^{*}\leftarrow {\mathit{x}}^K$ to all clients  ———————————————— 6: $\mathbf{ServerUpdate}(\mathit{x};\mathcal{I}):$ 7: send $\mathit{x}$ to clients $i\in \mathcal{I}$ 8: for each client $i\in \mathcal{I}$ in parallel do 9:       ${\mathit{x}}_i\leftarrow \mathbf{ClientUpdate}(\mathit{x};i)$ 10:     send ${\mathit{x}}_i$ to server 11: end for 12: return $\frac{1}{\left|\mathcal{I}\right|}{\sum }_{i\in \mathcal{I}}{\mathit{x}}_i$  ———————————————— 13: $\mathbf{ClientUpdate}(\mathit{x};i):$ 14: initialize ${\mathit{x}}_{i,j}\leftarrow \mathit{x}$ with counter $j=0$ 15: for$j=0,1,...,\tau -1$do 16:       ${\mathit{x}}_{i,j+1}\leftarrow {\mathit{x}}_{i,j}^k-\eta g_i\left({\mathit{x}}_{i,j}^k\right)$ 17: end for 18: return ${\mathit{x}}_{i,\tau }$

3.2. Hierarchical Federated Learning

As we discussed in the previous sections, performing cloud-based FL involves massive clients, but incurs high communication overhead, while performing edge-based FL only incorporates a limited number of clients with cheaper costs, incurring training performance loss, thus we introduce the hierarchical FL (HFL) for combining their advantages. Intuitively, we perform edge-based FL in parallel on several edge servers $\mathcal{M}$ and then aggregate their edge-level global model (or edge model for short) after E rounds client-edge communication at a cloud server. Each edge server $m\in \mathcal{M}$ communicates with clients ${\mathcal{N}}_m$ in its range, which yields a family of disjoint sets ${\left({\mathcal{N}}_m\right)}_{m\in \mathcal{M}}$, i.e., ${\mathcal{N}}_m\cap {\mathcal{N}}_{m^{\prime }}=\varnothing$, $\forall m,m^{\prime }\in \mathcal{M}$. We summarize the process as Algorithm 2. Indeed, in this framework, the global model is updated as Line 11 of Algorithm 2,

$$\begin{array}{cc}\hfill & {\mathit{x}}^{k+1}\leftarrow \frac{1}{\left|\mathcal{M}\right|}{\sum }_{m\in \mathcal{M}}{\mathit{x}}_m^{k,E},\hfill \end{array}$$

(4)

where ${\mathit{x}}_m^{k,E}$ represents the updated edge model, which is obtained by performing ServerUpdate operation on ${\mathit{x}}^k$ with the subset of clients ${\mathcal{N}}_m$E times in the k-th cloud communication round, as Line 4–10 of Algorithm 2. Therefore, the cloud server communicates with $\left|\mathcal{M}\right|$ edge servers rather than $\left|\mathcal{N}\right|$ clients, where the overhead is significantly reduced.

Algorithm 2 Hierarchical cloud–edge–client FL.

Input: local update step $\tau$, total communication round K, edge server set $\mathcal{M}$, fractions of clients ${\left({\mathcal{N}}_m\right)}_{m\in \mathcal{M}}$ Output: optimized global model ${\mathit{x}}^{*}$

1: cloud randomly initializes global model ${\mathit{x}}^k$ with counter $k=0$ 2: for each round $k=0,1,...,K-1$ do 3:       send ${\mathit{x}}^k$ to edge $m\in \mathcal{M}$ 4:       for each edge $m\in \mathcal{M}$ in parallel do 5:             initialize ${\mathit{x}}_m^{k,e}\leftarrow {\mathit{x}}^k$ with counter $e=0$ 6:             for each round $e=0,1,...,E-1$ do 7:                    ${\mathit{x}}_m^{k,e+1}\leftarrow \mathbf{ServerUpdate}({\mathit{x}}_m^{k,e};{\mathcal{N}}_m)$ 8:             end for 9:             send ${\mathit{x}}_m^{k,E}$ to cloud 10:       end for 11:       aggregate ${\mathit{x}}^{k+1}\leftarrow \frac{1}{\left|\mathcal{M}\right|}{\sum }_{m\in \mathcal{M}}{\mathit{x}}_m^{k,E}$ 12: end for 13: server returns ${\mathit{x}}^{*}\leftarrow {\mathit{x}}^K$ to all clients

4. A Lightweight Detection Mechanism for Hierarchical Federated Learning

The HFL system is promising to well balance training efficiency and performance, where in each training round, on the client i, the model $\mathit{x}$ is trained with the private dataset to minimize the local loss $f_i\left(\mathit{x}\right)$. However, due to the statistical heterogeneity between end devices and malicious tampering of raw data, the convergence direction of the updated local model may be significantly different from or even opposite to the convergence direction of the global model. Such clients participating in the model aggregation yield a negative impact on the FL performance and convergence. To ensure the effectiveness of the FL framework, it is necessary to detect and filter adverse clients. In this section, we propose a lightweight mechanism that can detect malicious attackers at the parameter level as Figure 1, which is introduced in detail subsequentially.

Executing additional detection tasks may increase the computation time. Especially the detection task is processed in a single server in the classical FL system. Note that in an HFL system, the task can be divided and offloaded to edge servers for execution, that is, each edge server processes its individual partial detection flow simultaneously, which can be a more inherently parallelizable paradigm for reducing physical training time. Formally, let the detector as $D:\mathcal{N}\to {\mathcal{N}}^{\prime }$, the whole detection task can be described as $D\left(\mathcal{N}\right)$, which is used to filter adverse clients in $\mathcal{N}$ and return a trustworthy subset ${\mathcal{N}}^{\prime }$. We aim to establish a group of detectors ${\left(D_m\right)}_{m\in \mathcal{M}}$ at the edge, which hold

$$\begin{array}{c}\hfill {\mathcal{N}}^{\prime }={\cup }_{m\in \mathcal{M}}{\mathcal{N}}_m^{\prime }={\cup }_{m\in \mathcal{M}}{D}_m\left({\mathcal{N}}_m\right):=D\left({\cup }_{m\in \mathcal{M}}{\mathcal{N}}_m\right)=D\left(\mathcal{N}\right)\end{array}$$

(5)

to replace a single detector at the cloud server. Thus, we can perform $D_m$ in parallel at each edge server m to accelerate the detection process.

To avoid introducing heavy computation costs in the detection phase, we let each edge server m equip a simple detector $D_m$, which directly calculates the cosine similarity, i.e.,

$$\begin{array}{c}\hfill \mathbf{similarity}(\mathit{x},{\mathit{x}}^{\prime })=\frac{\langle \mathit{x},{\mathit{x}}^{\prime }\rangle }{\parallel \mathit{x}\parallel \parallel {\mathit{x}}^{\prime }\parallel },\end{array}$$

(6)

between the previous edge model ${\mathit{x}}_m^{k,e}$ and currently uploaded client models ${\left({\mathit{x}}_{m,i,\tau }^{k,e}\right)}_{i\in {\mathcal{N}}_m}$ (or ${\left({\mathit{x}}_{m,i}^{k,e}\right)}_{i\in {\mathcal{N}}_m}$ in the absence of ambiguity) to evaluate whether the participator is working properly by a defined threshold t. Specifically, if $\mathbf{similarity}({\mathit{x}}_{m,i}^{k,e},{\mathit{x}}_m^{k,e})>t$, we claim the client can normally participate in this training round; otherwise, we drop the client to protect model-aggregation. Hence, the edge detector $D_m$ checks all local models uploaded from clients ${\mathcal{N}}_m$ and then returns a trusted subset of clients to the server. Thereafter, the edge server aggregates local models from filtered clients $D_m\left({\mathcal{N}}_m\right)$. The procedure is summarized as Algorithm 3. The whole architecture is the same as Algorithm 2, however with a surrogate ServerUpdate operation as shown in Lines 2–9. Especially, the additional detection process is represented in Line 8 and Lines 10-15, which is performed on each edge server m as the detector $D_m$ for checking clients ${\mathcal{N}}_m$.

Algorithm 3 R-HFL: Robust Hierarchical FL.

1: run Algorithm 2 with the surrogate $\mathbf{ServerUpdate}$ function: 2: $\mathbf{SurrogateServerUpdate}(\mathit{x};\mathcal{I}):$ 3: send $\mathit{x}$ to clients $i\in \mathcal{I}$ 4: for each client $i\in \mathcal{I}$ in parallel do 5:       ${\mathit{x}}_i\leftarrow \mathbf{ClientUpdate}(\mathit{x};i)$ 6:       send ${\mathit{x}}_i$ to server 7: end for 8: detect ${\mathcal{I}}^{\prime }\leftarrow D(\mathcal{I};t)$ 9: return $\frac{1}{|{\mathcal{I}}^{\prime }|}{\sum }_{i\in {\mathcal{I}}^{\prime }}{\mathit{x}}_i$  ———————————————— 10: $D(\mathcal{I};t)\equiv PCS(\mathcal{I};t):$ 11: ${\mathcal{I}}^{\prime }\leftarrow \varnothing$ 12: if$\frac{\langle {\mathit{x}}_i,\mathit{x}\rangle }{\parallel {\mathit{x}}_i\parallel \parallel \mathit{x}\parallel }{|}_{i\in \mathcal{I}}>t$then 13:       ${\mathcal{I}}^{\prime }\leftarrow {\mathcal{I}}^{\prime }\cup \left\{i\right\}$ 14: end if 15: return ${\mathcal{I}}^{\prime }$

5. Further Analysis on Partial Cosine Similarity

The proposed detector is named after the partial cosine similarity (PCS). In this section, we mathematically show that PCS can control the update deviation to effectively stabilize the HFL process to resist poisoning attacks. Then, we claim that the HFL training model with PCS converges to a stationary point of a weighted version of the original objective (1) in general non-convex cases. All relevant proofs are provided in the Appendix A, Appendix B and Appendix C.

Before introducing the pertinent main results, one can first review some important conditions in the proof of HFL convergence [16].

Assumption A1 

(smoothness). The gradient of each local loss $f_i$ is L-Lipschitz continuous, i.e.,

$$\begin{array}{c}\hfill \parallel \nabla f_i\left(\mathit{x}\right)-\nabla f_i\left({\mathit{x}}^{\prime }\right)\parallel \le L\parallel \mathit{x}-{\mathit{x}}^{\prime }\parallel ,\forall \mathit{x},{\mathit{x}}^{\prime }\in \mathrm{dom}{f}_i,\end{array}$$

(7)

hence, each edge loss $f^m:=\frac{1}{|D_m\left({\mathcal{N}}_m\right)|}{\sum }_{i\in D_m\left({\mathcal{N}}_m\right)}{f}_i$ in edge server m is consequently L-Lipschitz continuous.

Assumption A2 

(gradient divergence). For any $\mathit{x}$, the gradient divergence ${\delta }_i^m$ between each local loss $f_i$ and the corresponding edge loss $f^m{|}_{i\in {\mathcal{N}}_m}$ satisfies

$$\begin{array}{c}\hfill \parallel (\nabla f_i-\nabla f^m)\left(\mathit{x}\right)\parallel \le {\delta }_i^m,\end{array}$$

(8)

while the divergence ${\Delta }^m$ between each edge loss $f^m$ and the global loss f satisfies

$$\begin{array}{c}\hfill \parallel (\nabla f^m-\nabla f)\left(\mathit{x}\right)\parallel \le {\Delta }^m.\end{array}$$

(9)

Theorem 1 

(controlled deviation). The gradient deviation of $f^m$ in an edge communication round is controlled by the similarity threshold $t\in [-1,1]$, that is,

$$\begin{array}{c}\hfill \parallel \nabla f^m\left({\mathit{x}}_m^{k,e+1}\right)-\nabla f^m\left({\mathit{x}}_m^{k,e}\right){\parallel }^2\le L^2(\parallel {\mathit{x}}_m^{k,e+1}\parallel -\parallel {\mathit{x}}_m^{k,e}{\parallel )}^2-2(t-1)L^2\parallel {\mathit{x}}_m^{k,e+1}\parallel \parallel {\mathit{x}}_m^{k,e}\parallel .\end{array}$$

(10)

Remark 1. 

The theorem depicts that the deviation is controlled in a limited range about the threshold t, especially as t goes up, the deviation control becomes more strict. This also explains that excluding local models which may hurt the edge model ${\mathit{x}}_m^{k,e}$ (i.e., $\mathbf{similarity}({\mathit{x}}_{m,i}^{k,e},{\mathit{x}}_m^{k,e})$$\le t$) from the aggregation can stabilize the HFL training on the edge-side and then diffuse the impact to the cloud.

Theorem 2 

(convergence to a critical point of the weighted objective). Given that (i) each client i has a probability $p_i\in [0,1]$ of being dropped due to the PCS mechanism in each edge communication round, (ii) the local learning rate $\eta :=\eta \left(k\right)$ varies with the cloud communication round k and satisfies ${\sum }_{k=0}^{\infty }\eta \left(k\right)=\infty ,{\sum }_{k=0}^{\infty }{\eta }^2\left(k\right)=0$, then for any positive number ϵ, the average-squared gradient norm of $\tilde{f}$ has the following:

$$\begin{array}{c}\hfill \underset{K\to \infty }{lim}\frac{1}{{\sum }_{k=0}^K\eta \left(k\right)}\sum _{k=0}^K\eta \left(k\right){\parallel \nabla \tilde{f}\left({\mathit{x}}^k\right)\parallel }^2\le ϵ,\end{array}$$

(11)

where $\tilde{f}:=\frac{1}{\left|\mathcal{N}\right|}{\sum }_{i\in \mathcal{N}}(1-p_i)f_i$ is a weighted version of the original one (1).

Corollary 1 

(convergence bound according to the original loss). In terms of Theorem 2, suppose ${\left({\mathit{x}}^k\right)}_{k\le K}$ is the sequence generated by performing the R-HFL method, the minimal gradient norm of the original global objective f is bounded as the following:

$$\begin{array}{c}\hfill \underset{K\to \infty }{lim}\underset{k\le K}{min}{\parallel \nabla f\left({\mathit{x}}^k\right)\parallel }^2\le 2C_{p,\delta ,\Delta }+\frac{{2\left|\mathcal{N}\right|}^2}{{\left({\sum }_{i\in \mathcal{N}}(1-p_i)\right)}^2}ϵ,\end{array}$$

(12)

where $C_{p,\delta ,\Delta }:={\sum }_{i\in \mathcal{N}}{(\frac{1}{\left|\mathcal{N}\right|}-\frac{1-p_i}{{\sum }_{i\in \mathcal{N}}(1-p_i)})}^2{\sum }_{i\in \mathcal{N}}{({\delta }_i^m+{\Delta }^m)}^2{|}_{i\in {\mathcal{N}}_m}$ measures the deviation degree, which is determined by the learning system properties $p,\delta ,\Delta$.

Remark 2. 

Theorem 2 illustrates that processing the proposed R-HFL is equivalent to performing SGD on the weighted objective $\tilde{f}$ rather than the original one f, which shows that applying PCS causes the optimization objective drift to raise the difficulty of achieving the ideal training performance. Additionally, Corollary 1 further presents that the drift is mainly controlled by the drop probability ${\left(p_i\right)}_{i\in \mathcal{N}}$, where we have $p_i=0,\forall i\in \mathcal{N}$ such that $C_{p,\delta ,\Delta }=0$. Combining Theorems 1 and 2 and Corollary 1, notice that a small similarity threshold t decreases the detection strength while a large one increases the drop probability, thus properly setting t is crucial for achieving optimal performance.

6. Experiment Results

In this section, experiment results are presented and discussed. Firstly, the experimental setup is established. Then, the effectiveness of the proposed R-HFL is discussed under different types of attacks. Finally, the effects of the crucial factor, threshold t, on final training performance are analysed in detail.

6.1. Experiment Setup

We set 5 edge servers ($\left|\mathcal{M}\right|=5$) where each one communicates with 5 independent clients ($|{\mathcal{N}}_m|=5,\forall m\in \mathcal{M}$). Thus, the number of all participators is set to 25 ($\left|\mathcal{N}\right|=25$). Each client, holding heterogeneous raw data with the non-iid splits [16], is randomly assigned to any edge server. Specifically, each client’s dataset contains only two randomly proportioned classes of labels, which hold different numbers of samples.

6.1.1. Model and Dataset

The classical multilayer perceptron (MLP) is used as the basic ML model, where the corresponding loss function is set as cross-entropy. Testing accuracy is used as the primary performance metric. The popular public dataset MNIST is used to train ML models for the image recognition task with the HFL framework, which contains 70,000 gray-scale images of handwritten digits, 60,000 for training and the remaining 10,000 for testing.

6.1.2. Adversarial Attacks

We investigate the performance of the proposed R-HFL under different poisoning attacks [17,31] as follows.

(i)

Additive Noise of Model Parameter (ANMP) adds Gaussian noise to local model parameters, where the noise obeys $N(0,1)$. That is, for client i, any component $e_j^i$ of the local model parameters ${\mathit{x}}_i$ satisfy that $e_j^i\leftarrow e_j^i{+\xi |}_{\xi \sim N(0,1)}$, where ${\mathit{x}}_i=(e_1^i,\cdots ,e_j^i,\cdots ,e_d^i)$.

(ii)

Sign-Flipping of Model Parameter (SFMP) flips the sign of local model parameters, i.e., for client i, the local model parameters ${\mathit{x}}_i\leftarrow -{\mathit{x}}_i$.

(iii)

Additive Noise of Data Distribution (ANDD) adds Gaussian noise to local datasets, where the noise obeys $N(0,\sqrt{10})$, which is similar to ANMP.

In this paper, we consider that there are no completely trusted participators in the FL training process, i.e., all anonymous participating clients may contain malicious intent or suffer from malfunctions or interference in the process, hence all clients have a probability (defined as $prob$, $0\le prob\le 1$) to send the poisoned local model to the server in any edge communication round. Thus, $prob$ can quantitatively characterize the intensity of an attack.

6.1.3. Hyper-Parameters

The learning rate $\eta$ is initialized as $0.01$ with a decay coefficient $0.995$ each cloud round. The local training batch size is set to 20. The number of local training epochs $\tau$ is set to 10. The frequency of edge aggregations E is set to 5 per cloud round, while the number of total cloud aggregations K is set to 10, i.e., the total number of local training epochs on each client $KE\tau$ is 500, the total number of edge communication rounds on each edge server $KE$ is 50. Additionally, the attack intensity $prob$ and the PCS detection threshold t default to $0.5$.

6.2. Numerical Results

6.2.1. Performance Evaluation under Different Attacks

Figure 2 describes the global testing accuracy varying with cloud communication rounds as well as the corresponding averaged edge testing accuracy varying with edge rounds, used to depict the final training performance of our proposed method in various settings. The edge testing accuracy is jaggedly increasing due to statistical heterogeneity [13]. Specifically, as shown in (a)(b), in the Traditional FL, all clients directly interact with a single server regardless of communication costs, which represents the ideal performance upper bound. HFL/R-HFL depicts the results without attacks, which illustrates that our proposed PCS detection mechanism has no effect on the theoretical performance. For R-HFL under attacks and HFL under attacks, we find that the proposed R-HFL can effectively alleviate the negative impact of attacks (take ANMP as an example) compared to the original HFL algorithm. Moreover, as shown in (c)(d), we investigate the R-HFL global model performance under different intensities of attacks (take ANMP and SFMP as examples). As the $prob$ increases, the performance in each communication round uniformly degrades, which shows attack intensity unavoidably influencing the FL training. Additionally, combining (a) and (c) (or (b) and (d)), we find the degradation is much more limited in R-HFL versus the original HFL due to the detection mechanism.

6.2.2. Effect of the Key Factor: Threshold t

Figure 3 depicts the effect of the key factor threshold t on R-HFL performance in the two model-poisoning scenarios. As shown in (a)(b), it is observed that an appropriate set of t for the proposed R-HFL exists to reach the optimal performance under the ANMP attack. Specifically, when $t=0$, we show that the performance of R-HFL is only slightly better than the original one under the attack, which means PCS detectors miss many adverse models in filtering due to the relaxed threshold. When $t=0.5$, the performance is close to the original HFL without the attack, which means the detectors successfully filter all adverse models, but the performance degrades due to the decrease in the number of normal participators [16,36]. When t = 0.95, the performance further degrades, which means the detectors not only filter all adverse models, but also filter not a few normal local models with high data heterogeneity, especially in early training rounds. In (c,d) (under the SFMP attack), we obtain similar conclusions, but it is noticed that when $t=0$, the performance of R-HFL is unchanged from the case of $t=0.5$. It is illustrated that SFMP attacks cause larger gradient changes than ANMP attacks to some extent (in MNIST public dataset, when the noise obeys standard normal distribution) since apparent model difference results in relaxed feasible threshold t.

Figure 4 depicts the effect of the threshold t in the data-poisoning scenario. As shown in (a,b), one can note that when t ≤ 0.5, the performance of R-HFL degenerates to the original HFL under the ANDD attack, which shows the model difference is small in the scenario, hence the lower limit of the feasible detection threshold is higher than the previous two attacks. Moreover, when $t=0.65$, we show that the performance of R-HFL is even worse than the original one under the attack, which indicates that much adverse information is missed in the filtering process, and what is worse is that even if a small fraction of adverse models are filtered out, its performance gain will be offset by the reduction in the number of normal participating clients. The improvement occurs at $t=0.8$, as the threshold t becomes large, malicious information is totally dropped out. Although the reduction of participating customers slows down the convergence rate, the R-HFL model ends up with a much better global performance than the poisoning model.

Additionally,

Table 1. Testing and detection accuracy varying with PCS threshold t in different scenarios.

Scenario	-	$\mathit{t}=0$	$\mathit{t}=0.5$	$\mathit{t}=0.65$	$\mathit{t}=0.8$	$\mathit{t}=0.95$	$\mathit{t}=1$
without attacks (FL)	89.11 (-)	-	-	-	-	-	-
without attacks (HFL/R-HFL)	88.71 (-)	-	-	-	-	-	-
ANMP (HFL)	40.55 (-)	-	-	-	-	-	-
ANMP (R-HFL)	-	41.08 (71.04)	86.42 (≈100)	-	-	86.28 (98.24)	20.61 (≈50)
SFMP (HFL)	1.16 (-)	-	-	-	-	-	-
SFMP (R-HFL)	-	86.41 (≈100)	86.41 (≈100)	-	-	86.16 (98.16)	20.61 (≈50)
ANDD (HFL)	84.69 (-)	-	-	-	-	-	-
ANDD (R-HFL)	-	84.69 (≈50)	84.70 (≈50)	82.65 (83.48)	86.63 (≈100)	86.43 (98.64)	20.61 (≈50)

summarizes the testing accuracy under all scenarios in different settings, where the results show that the proposed R-HFL method has significant advantages over the baselines. Furthermore, we provide the corresponding detection accuracy in brackets, which indicates that the test accuracy is positively correlated with the detection accuracy.

7. Conclusions

In this paper, a new hierarchical cloud–edge–end collaboration-based FL method called R-HFL was proposed. The corresponding properties of stabilization and convergence were analysed and discussed, and relevant experiments were studied elaborately. Specifically, the hierarchical architecture was considered in the R-HFL framework to explicitly reduce the required cloud communication round from $KE$ to K during training. Moreover, as a distributed anomaly detection technique, PCS was proposed and applied in R-HFL to stabilize and improve performance under poisoning attacks. The experimental results are in agreement with the theoretical analyses, which convincingly demonstrate the validity of the proposed R-HFL. Future work can investigate how to expand the paper’s result to effectively enhance the Byzantine robustness of multilayered FL systems with high global information diffusion delays, and more generally, how to leverage global information to mitigate the local detection bias caused by data heterogeneity for better stabilization and convergence.