Defense Against Information Integrity Attacks in Federated IoT Systems Using Inertial Momentum-Aware IALM-RPCA

Abstract

While federated learning offers a decentralized approach to model training, ensuring the integrity of the information from each IoT client remains a challenge. This work delves into the dynamics of multi-stage federated learning, its susceptibility to information integrity attacks, and how to defend against such threats. A comprehensive understanding of data uncertainty and the challenges of poisoning attacks is discussed, laying a solid groundwork for the proposed defense mechanisms. At its core, this paper introduces a novel multi-stage federated learning model that segments the federated learning process into distinct phases with a novel approach of inertial momentum-aware Inexact Augmented Lagrange Multiplier Robust PCA with constant momentum factor and unaltered norm of the traditional one, each tailored to optimize for both efficiency and security. This robust framework is then tested against data injection-based poisoning attacks, using sparse noise, and demonstrates the effectiveness of the proposed recovery techniques like Robust PCA. Performance results highlight the resilience and efficiency of the introduced model with novel reconstruction algorithm, emphasizing the importance of this approach in real-world IoT settings. Data analysis, model summaries, and impacts of adversarial attacks further reinforce the findings, which are evaluated using rigorous statistical metrics and machine learning algorithms. The paper concludes by acknowledging its efficiency in detection and recovery from data poisoning attacks, improving robustness and data reconstruction in IoT environments while highlighting opportunities for further security enhancements.

1. Introduction

The era of the Internet of Things (IoT) has seen an increase in the connectivity of everyday devices, establishing a network that spans personal, public, and industrial domains. This interconnection has generated a massive flow of data, with devices continuously communicating, collecting, and transmitting information. While the increase in IoT devices represents an advancement in technology into the future, it has also presented a number of new challenges, particularly in the areas of data processing and learning methodologies [1].

These devices, despite their limited processing power, have strengthened the demand for advanced computational models capable of handling large datasets with minimal latency. Federated learning (FL) has emerged as an advanced paradigm that takes advantage of the distributed nature of IoT networks. By enabling collaborative model training while maintaining data localized, FL enhances privacy and optimizes the computational load on individual devices [2].

However, the decentralization that defines FL also introduces new vulnerabilities. The integrity of data is of utmost importance, as it is the fundamental basis that ensures the reliability and success of federated learning. Attacks that involve the injection of corrupted data by malicious actors represent a significant danger to the integrity of federated learning systems [3]. These attacks can hamper the learning process, lead to incorrect model updates, and ultimately compromise the entire learning infrastructure [4].

In light of the ever-increasing number of IoT devices and the ever-increasing dependence on FL, it is vital not only to address the issues that lie ahead but also to come up with effective countermeasures [5]. Recognizing the growing deployment of IoT devices and the reliance on FL for data processing and machine learning tasks, our study seeks to strengthen the FL framework against such threats. We propose to explore and refine methodologies that can enhance the robustness of FL against data uncertainties and information integrity attacks, particularly those that originate from the client side.

The goals of this work are designed to enhance the stability and security of the federated learning approach. By doing so, we aim to protect the cooperative nature of the learning process, particularly in the context of the rapidly expanding realm of IoT technologies. This effort is crucial to ensure that as more devices connect and communicate, they do so on a platform that’s resistant to disruptions and compromises.

The primary objectives of our research are threefold: firstly, to tackle the immediate challenges arising within the rapidly evolving domains of the IoT and federated learning; secondly, proposing a novel momentum-aware Inexact Augmented Lagrange Multiplier Robust PCA algorithm which is more efficient than the existing one; and, thirdly, to establish a benchmark for future innovation in secure and dependable decentralized learning ecosystems.

The key contributions of this research are as follows:

• We developed a multi-stage federated learning model in order to strengthen the robustness and efficiency of decentralized learning against client-side data poisoning attacks.
• We have proposed a novel approach to the Inexact Augmented Lagrange Multiplier (IALM)-RPCA [6] method by integrating the inertial momentum with an average constant momentum factor ($\beta$ = 0.5) without any tuning of the $\beta$ value persisting the trait of the traditional IALM -RPCA algorithm.
• As a countermeasure against information integrity attacks, specifically client-side data poisoning attacks, we developed a sophisticated defense strategy using the Augmented Lagrange Multiplication method. Our proposed method ensures the successful recovery of data after being corrupted which results in detecting client-side data integrity attacks in an efficient and effective manner by hybrid Inception–Transformer used as a global model in FL setting.
• The novel approach of IALM-RPCA has been validated through a set of extensive experiments considering a benchmark dataset known as N-BaIoT [7]. The novel approach of IALM RPCA exhibits a positive outcome over existing IALM RPCA method for reconstruction.

The remainder of the paper is organized as follows: Section 2 provides a review of related work, focusing on foundational studies on federated learning and the security vulnerabilities it faces. Section 3 outlines our methodology, including the proposed multi-stage model and defense mechanisms against poisoning attacks and a novel approach by introducing inertial momentum-aware IALM-RPCA method with constant momentum factor. Section 4 presents the results, offering insights into the dataset, confusion matrix analysis, impact of adversarial attacks, accuracy trajectories over epochs, performance metrics under varying scenarios, comparative analysis between conventional and novel process and overall observations. Section 5 discusses the relevance of federated learning, the significance of data reconstruction, comparison with previous works, and future research directions. Section 6 concludes the paper.

2. Related Work

This section is focused on federated learning, more specifically highlighting the inherent vulnerabilities of federated learning as well as its potential defense mechanisms against threats, with a particular emphasis on poisoning attacks. The covered area can be roughly divided into two categories: the preliminary studies on federated learning and the security challenges that are associated with it.

2.1. Foundational Work on Federated Learning

In recent years, federated learning has experienced considerable developments, which has resulted in the opening up of a wide variety of strategies and approaches that can be utilized to take advantage of this decentralized training paradigm. In the meantime, it has been leading the way in advancing the field of machine learning. McMahan et al. [8] established a benchmark for future research when they proposed a revolutionary method for training machine learning models without the use of centralized data repositories. This mechanism emphasized effective communication and paved the way for further investigations. This study was further developed by Konečný et al. [9], who investigated the communication challenges associated with federated learning and developed architectures that reduce those difficulties.

Yang et al. [10] explored the challenges and future directions in the area of federated learning, presenting a more comprehensive perspective on its potential. Also, Smith et al. [11] have presented a multi-task learning technique in federated settings, addressing the problems and potential solutions connected to ensuring privacy in such decentralized systems. Additionally, recent work by Lim et al. [12] on federated learning in mobile and edge computing environments has shed light on new challenges and opportunities in this domain. Bonawitz et al. [13] have made significant contributions to scalable federated learning, focusing on practical applications in large-scale systems. By sharing an additional model with each client, Tanmoy et al. [14] show an increase in the accuracy and fast convergence of the global model in FL setting in some cases compared to the existing FedAVG method. Later on, Tanmoy et al. [15] have evaluated accuracy and weight divergence of global model in FL setting for both model sharing and data sharing process.

The decentralization of data, despite its obvious challenge, comes with complications. Predicting data patterns across multiple nodes, each of which has its unique characteristics, has been a primary focus of the investigation. Bhagoji et al. provided a discussion on the challenges faced by poisoning attacks inside federated environments. Ref. [16], which is helpful. Further insights into optimizing federated learning algorithms are offered by Li et al. [17], focusing on enhancing model performance and efficiency.

2.2. Security Vulnerabilities in Federated Learning

The escalating use of federated learning has notably increased security concerns, particularly in the face of poisoning attacks. These attacks, aimed at disrupting the learning process through data manipulation, have been extensively analyzed by Biggio et al. [18]. Their work provides a deep dive into the systemic vulnerabilities of federated learning systems and the potential consequences of malicious activities. Recent developments in poisoning attacks, especially in IoT, have introduced new challenges, as discussed by Liu et al. [19]. They emphasized the complexity of security in IoT implementations of federated learning.

Recent advancements in the landscape of poisoning attacks, especially the emergence of “bi-level poisoning attacks,” have added a new layer of complexity, especially in IoT domains. Billah et al. [20] conducted an in-depth exploration of the implications of these sophisticated attacks on regression models, explaining their significant impact. Moreover, Zhang et al. [21] have brought to light various threats within the federated learning framework, emphasizing the urgent need for effective and robust security measures to counteract these challenges.

In response to the evolving security threats in federated learning, particularly in the context of IoT, researchers have proposed advanced defensive strategies. Wang et al. [22] and Wei et al. [23] have introduced comprehensive approaches to safeguard against data poisoning and model tampering. These strategies are crucial in maintaining the integrity and reliability of federated learning systems in increasingly complex and interconnected environments. The foundational work on Robust Principal Component Analysis (RPCA) by Candès et al. [24] has demonstrated that a corrupted data matrix can be decomposed into its underlying low rank structure and a sparse corruption component using Principal Component Pursuit. This work provided one of the fundamental formulations for recovering low-rank matrices with sparse and possibly arbitrarily large corruptions.

Moreover, to tackle the problem of corrupted matrices, the first paper that paves the way to use of ALM RPCA instead of traditional Robust PCA to mitigate the corrupted matrix was authored by Lin et al. [6]. Later, various optimization methods have made ALM RPCA more precise. Lin et al. [25] developed the Linearized Alternating Direction Method with Adaptive Penalty (LADMAP) algorithm, which successfully overcomes the structural constraints of traditional alternating direction method (ADM) algorithms by introducing a proximal term and an adaptive penalty rule. Zhu et al. [26] proposed a momentum-enhanced proximal ADMM framework for RPCA that improves convergence and robustness in nonconvex image- and video-processing applications. The paper authored by Wang et al. [27] introduced an inertial momentum-based optimization algorithm for SVD-free RPCA to improve computational efficiency and convergence for large-scale noisy data problems. Wu et al. [28] acquainted inertial proximal gradient methods with Bregman regularization for a class of nonconvex optimization problems. Xia et al. [29] showed that an inertial constraint term in RPCA optimization accelerates convergence and improves matrix recovery performance under mixed-noise environments by using a fractional function to approximate the $L_0$ norm and dynamic, non-linear thresholding. In the recent literature, Thamilarasu et al. [30] proposed a defense mechanism based on a recurrent neural network, called SpaceTime–Deep Similarity Defense (ST-DSD), which leverages spatial and temporal information to detect and prevent poisoning attacks in federated learning models. Ovi et al. [31] introduced confident federated learning, a defense framework against data poisoning attacks, that identifies and removes mislabeled training samples and detects malicious workers using neuron activation-based clustering. Olapojoye et al. [32] put forward FedECPA, a defense strategy based on scaling in blockchain-based federated learning to mitigate model poisoning attacks. The approach filters out clients with outlier model weights when aggregating and showed improved robustness.

Despite considerable academic research dedicated to unraveling the complexities of federated learning, a pronounced knowledge gap persists, especially concerning its inherent security vulnerabilities, like client-side poisoning attacks. This research endeavor is committed to addressing these vulnerabilities, aiming to illuminate the inherent weaknesses of federated learning systems. By doing so, it lays a foundational framework for future advancements designed to secure these systems against the nature of evolving threats.

This work highlights the critical need for ongoing vigilance and proactive innovation in security strategies. It underscores the importance of developing robust and adaptable defenses to safeguard federated learning models, particularly against sophisticated and dynamically changing threats. In doing so, this research not only seeks to bridge the current knowledge gap but also to pave the way for the creation of more robust and secure federated learning architectures, ensuring their efficiency and reliability in diverse and challenging environments.

3. Methodology

During this study, we have employed a wide range of distinct approaches to the implementation of investigations, all of which are described in further depth in the accompanying breakdown for this section. Detailed information regarding the procedures for multi-stage federated learning is supplied to protect the clients’ data from attacks on their information integrity. Clarification is then provided on the countermeasure technique we utilized to recover the poisoned data. This was done to remove any ambiguity that may have been caused. Discussion is also taking on regarding the amount of data uncertainty for the training data.

3.1. Overview of the Workflow

The methodology that we provide is founded on several essential steps, including data collection, the incorporation of federated learning, adding sparse noise, the processing of the data, the training of a federated model following the processing, and the subsequent strategies for data evaluation. Every one of these primary steps is capable of being broken down even further into deeper components. Figure 1 depicts the core process that we followed consistently to provide a greater understanding of our methodology. The next parts will go into greater depth regarding the complexities of each component.

A paradigm in the field of machine learning known as federated learning makes it possible to train algorithms across a large number of decentralized devices or servers, each of which stores its local data samples. When the goal is to construct machine learning models using data scattered across various devices, such as smartphones, without centralizing the data on a single server, it is crucially important that this strategy ignores the need to transfer data samples directly. This is essential when the purpose is to develop these models.

3.2. Operation of Federated Learning

The typical operation of the federated learning algorithm is described below. Here in Figure 2, a visual representation of FL process has been added.

• The initial model is trained on a central server, which may be initialized randomly or based on some previous knowledge.
• The most recent model is communicated to all of the devices (clients) that are actively participating in the federation.
• Every device generates an updated model by using the data from its local environment.
• The changes that were made locally are sent to the server. Users’ privacy can be protected by making these changes anonymous, random, or otherwise hard to understand.
• The server aggregates all of these modifications and creates a new global model, often by taking an average of them. To balance the contribution of each device, the updates can be weighted, for example by the number of local samples on each device.
• Steps 2 through 5 are carried out multiple times throughout several rounds until the performance of the model satisfies the specified requirements.

Federated learning is a very active research area and there are many different variants of the basic federated learning algorithm that are designed to address its various challenges. Some examples include the Federated Averaging (FedAvg) model that was proposed by McMahan et al., 2016 [9] and the SCAFFOLD model [33] that was proposed by Karimireddy et al., 2020.

It is important to highlight that even though federated learning can help protect privacy because it does not exchange raw data, it does not automatically assure full privacy or security. This is something that should be kept in mind. There is a chance that additional security measures, such as differential privacy [34] or secure multiparty computation, might be implemented to provide more robust assurances regarding the confidentiality of user information.

3.3. Poisoning Attack on Federated Learning and Recovery

Particularly in federated learning environments, poisoning attacks provide significant threats to machine learning models. They introduce inaccuracies into the training data, leading to incorrect pattern learning and inaccurate performance from the model. The nature of these attacks might be harmful. A successful recovery from such attacks requires attack detection, separation of affected data, data reconstruction, model re-training, and post-recovery analysis. In addition, an effective understanding of the potential strategies for attacks and the system architecture to enhance robustness is required.

3.3.1. Sparse Noise Introduction

During our research, we looked into a particular form of poisoning attack known as sparse noise, which is distinguished by the fact that it corrupts only a portion of the data and the magnitude is significantly high or low compared with the normal data distribution, and instances are sparse in nature. This allowed the structure of the dataset to be preserved while allowing the integration of the intended disruptions. This effectively simulated potential attacks in the real world on IoT client data.

In the considered threat model, an adversary is assumed to gain unauthorized access to the training data before it is distributed among the participating IoT clients. The attacker can modify a sparse subset of entries in the dataset while leaving the majority of the data unchanged. The objective of the attacker is to introduce corrupted information into the training process and consequently degrade the reliability and performance of the federated learning model.

In our experimental setup, the attack is implemented by adding sparse Gaussian noise into randomly selected entries of the full dataset. Then the corrupted dataset is fed to the proposed IALM-RPCA-based reconstruction method to separate the underlying low-rank information from the sparse corruption and recover the original data structure. The reconstructed dataset is then divided and distributed to the federated learning clients that participate in the federated learning process.

We assume the attacker does not have control over the federated aggregation server and the federated learning algorithm itself. The attack is conducted at the data level, prior to the dataset distribution to the involved clients. Therefore, the proposed defense is a pre-training data integrity tool instead of a client- or server-side model update defense.

The provided algorithm, termed as Sparse Noise Introduction—Poisoning Attack, is described in Algorithm 1.

The algorithm begins with an initialization phase where a sparsity level is set, for example 0.05 shown in Algorithm 1, indicating the proportion of the dataset’s entries that will be targeted with noise. Based on this sparsity level, it calculates the number of noisy entries by multiplying the dataset’s total size by the sparsity factor. Next, it randomly selects indices from the data set equal to the number of noisy entries identified.

For each of these indices, Gaussian noise is generated, following a normal distribution with a mean of zero and a standard deviation of five. Here in Figure 3 the probability density function is shown. The algorithm then applies this noise to the dataset, altering the original entries at the chosen indices to simulate the effect of an information integrity attack.

The outcome is the data set with the the noise introduced in a sparse manner, indicating a poisoning attack. To pursue the effect of attack irrespective of the different type of feature magnitude, so that for any later comparison of the two algorithms the real insight could be reflected, we have used the standard calling method after mixing raw noise with the data set. So, keeping the shape of the noisy dataset similar, all features were equally weighted so that, later on, RPCA could treat the value of all features fairly.

Figure 4 presents the distributions of several representative features. Although Gaussian noise was added randomly to entries throughout the entire feature matrix, rather than to each feature individually, the resulting distributions of the selected features in Figure 5 become bell-shaped and approximately Gaussian. This demonstrates the impact of the corruption process on the underlying data distribution.

Now, to enable effective low-rank and sparse decomposition by RPCA, we have scaled the data. In standardization, the usefulness of scaling is evident as it ensures equal feature contribution to the nuclear norm minimization and allows the regularization parameter to work uniformly across all features, leading to unbiased recovery of the low-rank structure and consistent sparse anomaly detection with efficient IALM convergence.

Here in Figure 6 only the feature values have been rescaled without changing the shape and noise structure. The standardization merely shifts each feature to zero mean and unit variance by a linear transformation, preserving the underlying data distribution and the additive Gaussian noise pattern. This is important in order for the noise characteristics to remain consistent and the low-rank and sparse structures required by RPCA to be preserved for faithful reconstruction and anomaly detection.

Algorithm 1: Sparse Noise Introduction—Poisoning Attack

It is essential to recognize that this method alters quite a few of the entries in the dataset with noise based on the Gaussian distribution. The subsequent analyses or performances of machine learning models may be dramatically impacted as a result of this.

3.3.2. Reconstruction of Noisy Data Using RPCA

The following section will provide a more in-depth breakdown of the procedures that are utilized for Robust Principal Component Analysis (RPCA) [35] and matrix completion. The large variety of RPCA approaches include methods like Accelerated Proximal Gradient, Dual Method, Singular Value Thresholding, Alternating Direction Method, and ALM (Augmented Lagrange Multiplier), among others. The Inexact ALM [35] approach has emerged as the leading choice among them, demonstrating speed and accuracy that are superior to any other method. As a result, its position as the favored alternative for data reconstruction in our research has been enhanced as a result of this development. As can be deduced from the names of the procedures, certain strategies, such as the Augmented Lagrange Multiplier and the Singular Value Thresholding, have been shown to be effective at completing matrices.

3.3.3. Inexact ALM RPCA Algorithm

Consider a matrix $M\in {\mathbb{R}}^{m\times n}$. The Inexact ALM RPCA algorithm addresses the optimization problem:

$$\underset{L,S}{\min }{\parallel L\parallel }_{*}+\lambda {\parallel S\parallel }_1\mathrm{subject}\mathrm{to}L+S=M,$$

(1)

where ${\parallel ·\parallel }_{*}$ represents the nuclear norm (i.e., the sum of singular values) and ${\parallel ·\parallel }_1$ denotes the ${\ell }_1$ norm (i.e., the sum of absolute values). Here, L and S are low-rank and sparse matrices, respectively.

The algorithm procedure is

• Initialize: $Y=0$, $S=0$, and $L=0$. Set $\mu ={\displaystyle \frac{1.25}{{\parallel M\parallel }_2}}$, $\rho =1.5$, and ${\max }_{\mu }={10}^{10}$.
• Repeat until convergence:

  (a)

  Update S using the shrinkage operator: $S=\mathrm{shrink}(M-L+Y/\mu ,\lambda /\mu )$.

  (b)

  Update L with the Singular Value Thresholding (SVT) operator:

  $$\begin{array}{cc}\hfill L& =\mathrm{SVT}(M-S+Y/\mu ,1/\mu ),\hfill \\ \hfill \mathrm{where}U\mathsf{\Sigma }{V}^T& =\mathrm{svd}\left(X\right),\hfill \\ \hfill \mathsf{\Sigma }& =\mathrm{diag}\left(\mathrm{shrink}\right(\sigma ,\tau \left)\right).\hfill \end{array}$$

  (c)

  Update Y: $Y=Y+\mu (M-L-S)$.

  (d)

  Update $\mu$: $\mu =\min (\rho \mu ,{\max }_{\mu })$.

• Convergence is achieved when ${\parallel M-L-S\parallel }_F/{\parallel M\parallel }_F<\mathrm{tol}$, with ’tol’ as a pre-defined threshold.

3.3.4. Inertial Momentum-Aware Inexact ALM RPCA Algorithm

In the case of inertial momentum-aware IALM RPCA, all the steps of traditional IALM RPCA are consistent except for the addition of constant momentum factor ($\beta$ = 0.5) and extrapolation step of finding $L_k$ using $L_{k-1}$ and $L_{k-2}$ of the previous step for each kth iteration until convergence. We have taken into consideration that beta should be an average value because, if $\beta$ = 1; the full difference affects the current value of the denoised, sparse matrix which will make the current result backward and, in the case of a negative momentum parameter $\beta$ < 0, results in the opposite of momentum, which is of no use. And if $\beta$ = 0, that leads to traditional IALM RPCA.

Here is the algorithm procedure:

• Initialize: $Y=0$, $S=0$, and $L=0$. Set $\mu ={\displaystyle \frac{1.25}{{\parallel M\parallel }_2}}$, $\rho =1.5$, ${\max }_{\mu }={10}^{10}$ and $\beta$ = 0.5.
• Repeat until convergence:

  (a)

  $\overline{L}$ = $L_{k-1}$ + $\beta$*($L_{k-1}$ - $L_{k-2}$) ← addition of inertial momentum

  (b)

  Update $S_k$ using the shrinkage operator: $S_k=\mathrm{shrink}(M-\overline{L}+Y/\mu ,\lambda /\mu )$.

  (c)

  Update $L_k$ with the Singular Value Thresholding (SVT) operator:

  $$\begin{array}{cc}\hfill L_k& =\mathrm{SVT}(M-S_k+Y/\mu ,1/\mu ),\hfill \\ \hfill \mathrm{where}U\mathsf{\Sigma }{V}^T& =\mathrm{svd}\left(X\right),\hfill \\ \hfill \mathsf{\Sigma }& =\mathrm{diag}\left(\mathrm{shrink}\right(\sigma ,\tau \left)\right).\hfill \end{array}$$

  (d)

  Update Y: $Y=Y+\mu (M-L_k-S_k)$.

  (e)

  Update $\mu$: $\mu =\min (\rho \mu ,{\max }_{\mu })$.

• Convergence is achieved when $\parallel M-L_k-S_k{\parallel }_F/{\parallel M\parallel }_F<\mathrm{tol}$, with ’tol’ as a pre-defined threshold.

We have used the traditional IALM RPCA algorithm with inertial momentum factor. We have not changed the ${\ell }_1$ norm and thresholding process, rather we preserve the norm type. Keeping all traits of IALM RPCA unaltered, we have used the extrapolation method to introduce the inertial momentum factor. Moreover, we have assumed an average momentum factor $\beta$ = 0.5 whereas a paper authored by Xia et al. [29] have used grid search to find beta and other parameters. Grid search can become very time-consuming, especially when many parameters exist, datasets are large, and each training or optimization run is expensive.

3.4. Overview of Used Multi-Stage Federated Learning Model

Within the scope of this study, the application of the Federated Averaging (FedAvg) algorithm was applied. The FedAvg algorithm [9], which was proposed by McMahan et al. in 2016, has now developed into a standard method in federated learning.

The following is an outline of the primary processes that make up the FedAvg algorithm:

• Initialization: The server initializes a global model $w_0$.

  $$w_0\leftarrow \mathrm{Initialization}\mathrm{of}\mathrm{the}\mathrm{global}\mathrm{model}$$
• Model Distribution: The server sends the global model $w_t$ to each of the K clients.

  $$\mathrm{Send}{w}_t\mathrm{to}\mathrm{each}\mathrm{of}\mathrm{the}\mathrm{K}\mathrm{clients}$$
• Local Training: Each client k computes an updated model based on its own local data $D_k$. Each client performs E epochs of SGD with batch size B on its local dataset to compute the update.

  $$w_{t+1}^k\leftarrow \mathrm{SGD}(w_t;D_k)$$
• Local Model Upload: Each client sends its model updates back to the server.

  $$\mathrm{Send}{w}_{t+1}^k\mathrm{to}\mathrm{the}\mathrm{server}$$
• Global Aggregation: The server aggregates these updates to form a new global model. This is done by taking a weighted average of the clients’ updates, where the weights could be proportional to the number of data points each client has.

  $$w_{t+1}\leftarrow {\displaystyle \frac{1}{{\sum }_{k=1}^K{n}_k}}{\displaystyle \sum _{k=1}^K}{n}_k{w}_{t+1}^k$$
• Repeat Steps 2–5: This process is repeated for several rounds until the model performance meets the desired criteria.

In this part, a detailed description of the research methodologies that were used in this investigation was provided. The mathematical foundations of the robust variation of Principal Component Analysis (PCA), which is referred to as Robust PCA, has been investigated and evaluated by this study. The Principal Component Analysis (PCA) is subject to severe erroneous points or outliers in the data, which is why many people choose the robust form of the Principal Component Analysis (PCA) [6]. In order to solve this problem, a number of different procedures for doing robust PCA were outlined [6]. In terms of the rank of the estimate, the relative error, and the amount of processing time that was necessary, each of these algorithms had its own unique set of benefits and drawbacks.

A lot of effort and consideration has already been given to various kinds of RPCA algorithm to make them more precise and robust. We have tried to make the IALM RPCA method more robust and efficient without changing it’s core methodology which displayed a high level of efficiency and performance over the traditional one. Following the breakdown of the algorithm into its component parts and subsequent expression using mathematical notation, the readability of the algorithm was improved significantly.

In order to correctly reconstruct the data, both traditional IALM RPCA and inertial momentum-aware IALM RPCA were applied, resulting in revealing the robustness of a novel approach of IALM RPCA, and, after that, the research endeavored to investigate federated learning. This cutting-edge method of machine learning allowed for the protection of users’ privacy concerns while simultaneously facilitating the creation of models that were founded on reconstructed data. Due to the fact that it enables the construction of models without the prerequisite of having to provide raw data, this method is significant when dealing with sensitive data.

The methodologies and algorithms that have been discussed up to this point served as the basis for the tests and results that will be discussed in the following section. The performance of each method on the dataset under consideration will offer us insights into the method for dealing with outliers in PCA that is the most effective and efficient based on those findings. In conjunction with federated learning, the objective of this research is to develop a method of data analysis that is robust, efficient, and protective of users’ privacy even when dealing with erroneous or infected data. This will be accomplished by developing a technique for data analysis that is robust, efficient, and protective of users’ privacy.

3.5. Global Model Architecture

The global model, a hybrid Inception–Transformer architecture intended for network traffic categorization, was cooperatively trained by multiple clients without exchanging raw data. Figure 7 shows the detailed model architecture and Figure 8 demonstrates the process flow diagram. The goal is to learn discriminative patterns from network flow data in order to differentiate benign traffic and various attack types.

The model uses a Transformer encoder with a Multi-Head Self-Attention (MHSA) layer to extract long-range dependencies in traffic features, which consists of three attention heads, each of dimension 64. We use residual connections and layer normalization to keep the feature representations and to stabilize the training. Instead of a conventional feed-forward sublayer, we replace the feed-forward component in the Transformer block with three stacked Inception blocks that extract multi-scale local patterns. Each Inception block consists of four parallel branches: (i) a convolutional branch with kernel size 10, (ii) a two-stage convolutional branch with kernel sizes 10 and 30, (iii) a two-stage convolutional branch with kernel sizes 10 and 50, and (iv) a max-pooling branch followed by a convolutional layer. The four branches use 16, 24, 12, and 8 filters, respectively.

The outputs of the parallel branches are concatenated and then passed through batch normalization and ReLU. Then we use a dropout with rate 0.1 for regularization and a 1 × 1 convolutional layer to project the output features to the required dimensionality. The output of the attention is added to the result of layer normalization with a residual connection. Then Global Average Pooling is used to get a fixed length feature representation. This representation is fed to a full connected layer of 32 neurons with ReLU activation, followed by dropout regularization. Finally, a softmax layer outputs the class probabilities for benign traffic and the attack classes considered.

This architecture effectively learns both contextual and local discriminative patterns by fusing the multi-scale extraction capability of Inception modules with the global dependency modeling capability of Transformer attention. This makes it appropriate for multi-class attack classification and federated network intrusion detection.

3.6. Feature Selection Process

The aim of the feature selection process was to decrease the input dimensionality without losing the most informative features. To assess the importance of each feature, an Extra Trees classifier consisting of 50 trees was trained on the training data, and the importance scores obtained were then used to rank the features, keeping only those whose importance was greater than the average importance. The feature selection procedure was constructed using only the training data and then applied to both the training and the test datasets to ensure consistency and prevent data leakage. The resulting feature sets were reshaped into a three-dimensional format, (N, F, 1), where N stands for the number of samples and F for the number of selected features, before being given as input to the neural network.

3.7. Global Model Training and Update

For federated training, the dataset is distributed across 10 simulated clients with the IID (random balanced) sampling strategy and each client is trained on mini-batches of size 64. Each client trains a local copy of the global model for one epoch per communication round with the Adam optimizer (default learning rate) and categorical cross-entropy as the loss function; class-weighted loss is used during local training. Then, the locally trained client models are uploaded to the server and their updates are scaled and aggregated by a weighted aggregation strategy similar to FedAvg to obtain the updated global model in each round. Hence, the global model is updated by aggregating locally trained models.

4. Results

The findings of our research shed insight into the discovery of the novel approach of adding a momentum term to IALM RPCA using the extrapolation method and combining with federated learning; it shows the efficiency as well as the robustness across a wide variety of circumstances. These findings demonstrate both the limitations and strengths that are inherent in federated learning models, and they also indicate the critical role that data reconstruction plays in reducing the vulnerabilities that are emphasized. In addition, these findings highlight the crucial role that data reconstruction plays in reducing the vulnerabilities that are highlighted using less computation and time.

4.1. Datasets

In our research, we utilized the ‘N-BaIoT Dataset’ [7] available in the UCI Machine Learning Repository. This dataset was developed in response to the notable absence of publicly accessible botnet datasets specifically tailored for the IoT domain. It encompasses real traffic data extracted from nine commercial IoT devices that were genuinely compromised by renowned malware, such as Mirai and BASHLITE. Figure 9 illustrates the dataset’s classes along with their respective value counts, including ten ’attack’ types and one ‘benign’ class.

Table 1. Dataset characteristics.

Characteristic	Detail
Type	Multivariate, Sequential
Instances	7,062,606
Attributes	115 (Real Number Type)
Tasks	Classification, Clustering
Date Donated	19 March 2018

provides a detailed overview of the key attributes of the dataset employed in our study. It is structured into two columns: ‘Characteristic’ and ‘Detail’, offering a clear distinction between the attributes and their specifics. The dataset is categorized as ‘Multivariate, Sequential’, suggesting its complexity and sequential nature. It encompasses an extensive collection of 7,062,606 instances, each described by 115 real-number attributes. This dataset is versatile, as indicated by its applicability in both ‘Classification’ and ‘Clustering’ tasks. The inclusion of the ‘Date Donated’ offers insight into the dataset with a donation date of 19 March 2018.

Although the primary purpose of this dataset was to differentiate between benign and malicious traffic through the use of anomaly-detection techniques, the multifaceted nature of the malicious data enables it to also be used for multi-class classification. This is due to the fact that the malicious data consists of 10 different attacks that were initiated by two different botnets. The full N-BaIoT dataset contains traffic from nine devices, but we only use samples corresponding to device 5. In particular, we use the 5.benign.csv file with the available 5.gafgyt.* and 5.mirai.* files. This resulted in the numerical results presented in this work being obtained from the selected Device-5 subset instead of the entire nine-device dataset.

For further insights and dataset access, consult the UCI Machine Learning Repository at the following URL: http://archive.ics.uci.edu/ml/datasets/detection_of_IoT_botnet_attacks_N_BaIoT (accessed on 12 February 2024). A deeper exploration of the dataset and related research can be undertaken on Kaggle at https://www.kaggle.com/datasets/mkashifn/nbaiot-dataset (accessed on 10 February 2024).

4.2. Confusion Matrix Insights

Figure 10 visualizes the classification capabilities of our model. A noticeable trend is the high true positives under a “No Attack” context. However, adversarial interventions significantly lowered true classifications. A marked improvement in true positives was observed post-reconstruction.

4.3. Impact of Adversarial Attacks

Different levels of poisoning were tested in order to determine how well our federated learning strategy performed in a variety of information integrity attacks. The following

Table 2. Evaluation of the federated learning approach under various information integrity attack scenarios.

Poisoning Rate	Clients (K)	1st Round			10th Round
		B	W	G	B	W	G
0% (No Attack)	10	55.28	52.58	73.11	82.51	81.67	82.74
5%	10	27.32	25.68	64.09	52.75	51.31	69.01
10%	10	18.18	17.50	55.86	40.82	39.57	60.88
15%	10	14.70	14.08	53.30	34.03	32.44	58.88

* B: best client accuracy. W: worst client accuracy. G: global model accuracy.

presents a comprehensive summary of the performance of the federated learning model when subjected to a variety of information integrity attack scenarios.

Table 2 provides a comprehensive evaluation of the federated learning approach under varying information integrity attack scenarios. The scenarios are determined based on different poisoning rates ranging from 0%, signifying no attack, to 15%, indicating that 15% clients’ data were subjected to poisoning.

Each scenario comprises:

• The number of clients, uniformly represented as K = 10.
• The accuracies observed during ten distinct training rounds: the initial (1st round) to the last stage (10th round).

For each of these rounds, the performance metrics are split into three categories:

• B (Best Client Accuracy): This depicts the highest accuracy achieved by any single client.
• W (Worst Client Accuracy): This represents the lowest accuracy across all clients.
• G (Global Model Accuracy): This metric illustrates the accuracy of the federated global model after aggregating updates from all clients.

One may see how the integrity attacks affect the performance of the federated learning model by looking at Table 2. For instance, when there is no client attack present (the poisoning rate is set to 0%), the global model reaches an accuracy of 73.11% in the first round and significantly improves to 82.74% by the tenth round. It is important to note the model’s robustness and its flexibility to adversarial strategies as the poisoning rate increases. This is clear from the variances in accuracy across different poisoning rates.

According to Table 2, it is clear that the federated learning model maintains a noteworthy performance even when adversarial attacks are taken into consideration, and this is especially true when measured against the global accuracy metric.

4.4. Accuracy Trajectory over Epochs

As is clear in Figure 11, the model’s learning trajectory is central to comprehending its adaptability. Notably, even in the context of adversarial challenges, the rate of convergence remained noteworthy, particularly after the introduction of the data reconstruction mechanism. In Figure 11 above, it is clearly seen that the dotted line representing the IALM RPCA with inertial momentum approach outperforms the traditional IALM RPCA approach in all levels of attack, and the with no attack the model performance is undoubtedly the best one.

4.5. Performance Metrics Under Varying Scenarios

The model’s performance metrics across different scenarios, no attack, post-attack, and post-reconstruction by two approaches, are consolidated in Figure 12. It’s clear that adversarial attacks notably impaired the performance, but the data reconstruction process played a pivotal role in increasing the model’s efficiency and the inertial momentum-aware IALM RPCA with constant momentum factor is giving a better performance than existing IALM RPCA.

4.6. Comparative Analysis of Performance Metrics

The comparisons of different evaluation metrics for noisy data reconstruction are demonstrated here in

Table 3. Performance metrics of the FL model for 25% attack.

Metric	No Attack	Post Attack	Post-Reconstruction		Improvement
			IALM	IALM (Mom.)	IALM	IALM (Mom.)
Accuracy	83.34%	20.00%	76.73%	79.54%	56.73%	59.54%
Precision	80.89%	21.10%	73.37%	76.81%	52.27%	55.71%
Recall	86.16%	21.45%	79.51%	82.19%	58.06%	60.74%
F1-score	81.83%	12.65%	74.97%	77.68%	62.32%	65.03%

,

Table 4. Performance metrics of the FL model for 50% attack.

Metric	No Attack	Post Attack	Post-Reconstruction		Improvement
			IALM	IALM (Mom.)	IALM	IALM (Mom.)
Accuracy	83.34%	21.40%	77.58%	79.20%	56.18%	57.80%
Precision	80.89%	16.63%	74.55%	76.11%	57.92%	59.48%
Recall	86.16%	23.09%	80.41%	82.00%	57.32%	58.91%
F1-score	81.83%	12.74%	75.73%	77.45%	62.99%	64.71%

,

Table 5. Performance metrics of the FL model for 75% attack.

Metric	No Attack	Post Attack	Post-Reconstruction		Improvement
			IALM	IALM (Mom.)	IALM	IALM (Mom.)
Accuracy	83.34%	18.84%	77.63%	79.27%	58.79%	60.43%
Precision	80.89%	11.62%	74.05%	76.79%	62.43%	65.17%
Recall	86.16%	17.41%	80.23%	81.77%	62.82%	64.36%
F1-score	81.83%	8.04%	75.90%	77.27%	67.05%	69.23%

and

Table 6. Performance metrics of the FL model for 100% attack.

Metric	No Attack	Post Attack	Post-Reconstruction		Improvement
			IALM	IALM (Mom.)	IALM	IALM (Mom.)
Accuracy	83.34%	23.70%	74.82%	75.19%	51.82%	51.49%
Precision	80.89%	11.72%	72.78%	72.97%	61.06%	61.25%
Recall	86.16%	26.13%	77.55%	76.53%	51.42%	50.4%
F1-score	81.83%	15.00%	73.25%	72.64%	58.25%	57.64%

. Here, different types of attack scenario have been taken into consideration to measure the robustness of each algorithm. The improvement column of each table states that.

This comparative evaluation shows that the post-reconstruction performance came close to approaching the no-attack baseline, which highlights the efficiency of the data reconstruction method that we employed. Surprisingly, even in the presence of a threat condition, the reconstructed model displayed a strong recovery, reaching an accuracy of near to 80% that outperformed the accuracy of the traditional IALM approach, which was only a bit behind the 83.34% that was seen in the case where there was no attack.

4.7. Comparative Analysis of Reconstruction Time

To compare time, reconstruction has been done under different attack scenarios 20 times and the average, median, max time and min time are recorded.

The IALM with Momentum methodology consistently needs more computing time than the normal IALM method across all attack percentages, as shown by the reconstruction time analysis in

Table 7. Comparative analysis of reconstruction time (sec) under different attack percentages.

Attack (%)	Min Time		Max Time		Average Time		Median Time
	IALM	IALM (Mom.)	IALM	IALM (Mom.)	IALM	IALM (Mom.)	IALM	IALM (Mom.)
25%	508.08	538.47	573.27	654.05	524.35	557.34	519.13	548.285
50%	452.50	491.08	475.82	514.78	460.31	499.12	459.89	496.27
75%	465.23	506.76	512.35	568.31	489.59	533.00	494.06	537.00
100%	464.90	506.06	521.25	560.12	491.50	535.72	498.82	546.14

. For example, the average reconstruction time went from 524.35 s for IALM to 557.34 s for IALM with Momentum under the 25% attack scenario. The momentum-based variation exhibits increased minimum, maximum, average, and median reconstruction durations for 50%, 75%, and 100% attack settings.

The momentum mechanism, which adds an extra update component during optimization, is responsible for this rise in computing cost. This approach helps to increase optimization stability and refine low-rank reconstruction even if it slows down the reconstruction process. As a result, although IALM with Momentum takes longer to execute, it achieves better reconstruction quality, which improves classification performance over normal IALM in terms of accuracy, precision, recall, and F1-score.

As a result, there is a trade-off between reconstruction efficiency and computational efficiency. While IALM with Momentum compromises more computing time to obtain higher overall prediction performance and robustness against attacks, standard IALM delivers quicker execution. But in the case of good Cyber-Physical System (CPS) attack, removing is much more important than time efficiency, because if the attack remains it can result in rigorous loss in the long run.

4.8. Performance Assessment of Various $\beta$ Values at Different Poisoning Attack Intensities

For each attack level, comparison of the different $\beta$ values and identification of the best performer has been performed here.

Regarding the impact of the inertial momentum parameter $\beta$, a distinct pattern becomes apparent. The optimal balance between robustness and convergence speed is consistently achieved with moderate values of $\beta$ (0.5) in most cases (highlighted in

Table 8. Performance under 25% poisoning attack.

$\mathit{\beta }$	Accuracy	Precision	Recall	F1-Score
−0.5	76.01	72.56	78.73	74.14
−0.3	77.53	74.60	80.70	75.90
−0.1	77.08	74.08	80.24	75.65
0.1	78.01	74.30	79.98	75.85
0.3	78.06	75.36	81.50	76.45
0.5	79.54	76.81	82.19	77.68
0.7	79.62	76.78	82.14	77.72
0.9	78.01	75.70	80.55	76.25
1	72.20	66.50	69.40	66.89
1.2	23.42	4.44	14.30	6.53

,

Table 9. Performance under 50% poisoning attack.

$\mathit{\beta }$	Accuracy	Precision	Recall	F1-Score
−0.5	75.87	72.81	78.87	74.33
−0.3	76.50	73.64	79.59	74.76
−0.1	77.59	74.94	80.80	75.96
0.1	78.18	75.87	81.53	76.67
0.3	79.28	76.58	82.10	77.75
0.5	79.20	76.11	82.00	77.45
0.7	78.24	75.69	81.66	76.68
0.9	77.55	74.52	80.52	75.78
1	63.36	58.00	60.25	55.48
1.2	23.40	4.62	14.29	6.63

,

Table 10. Performance under 75% poisoning attack.

$\mathit{\beta }$	Accuracy	Precision	Recall	F1-Score
−0.5	76.40	73.41	79.72	74.82
−0.3	78.64	75.19	80.90	76.65
−0.1	78.62	75.78	80.88	76.36
0.1	79.28	76.34	81.58	77.15
0.3	78.66	75.75	81.75	76.97
0.5	79.27	76.79	81.77	77.28
0.7	76.89	73.89	79.81	74.96
0.9	78.57	76.68	81.25	77.03
1	69.43	63.18	67.77	62.97
1.2	23.43	4.56	14.30	6.60

and

Table 11. Performance under 100% poisoning attack.

$\mathit{\beta }$	Accuracy	Precision	Recall	F1-Score
−0.5	74.17	71.26	76.04	72.57
−0.3	74.04	71.94	76.64	72.39
−0.1	73.89	70.06	74.81	71.48
0.1	74.54	71.23	76.32	72.09
0.3	71.55	68.08	71.49	67.65
0.5	75.19	72.97	76.53	72.64
0.7	79.18	75.81	81.72	77.22
0.9	77.64	76.09	80.34	75.69
1	69.75	58.87	65.40	61.15
1.2	11.68	1.26	9.09	2.20

). In particular, $\beta$ = 0.7 exhibits higher resistance in the most severe 100% assault scenario, achieving an F1-score of 77.22%, whereas $\beta$ = 0.3 performs best under a 50% poisoning attack. Although $\beta$ = 0.5 does not always achieve the highest score, it maintains stable performance across all attack intensities, indicating strong generalization. In general, extremely tiny momentum ($\beta$ = 0.1) and very large momentum ($\beta$ = 0.9) result in worse performance, indicating that the quality of RPCA reconstruction may be adversely affected by either insufficient or excessive inertial effect. We also evaluated the performance using negative values of $\beta$. The experimental results indicate that values of $\beta$ greater than 1 lead to a gradual degradation in performance, while negative values of $\beta$ also fail to outperform the stable performance achieved with $\beta$ = 0.5 in all types of poisoning scenario. Moreover, as a small amount of carefully prepared poisoned data can have disproportionately large negative consequences for large language models, with recent experiments showing that a near-constant number of poisoned documents can compromise models of significantly different model and dataset sizes [36], efficient reconstruction of sparsely poisoned data is crucial for enabling effective detection and mitigation of such attacks. In this context, the findings show that the strongest resistance against data poisoning attempts in most cases is offered by the $\beta$ value 0.5, particularly under low-sparsity poisoning conditions.

5. Discussion

The results of our study highlight the inherent potential of federated learning (FL) in assuring model robustness, particularly when it is set to the test in adversarial situations. The adaptability of the FL approach is further shown by the introduction and subsequent effectiveness of data reconstruction techniques, which is highlighted by the findings.

5.1. Relevance of Federated Learning

The findings of our study provide strong evidence on the efficiency of federated learning, demonstrating its robustness not only in typical scenarios but also in the face of adversarial conditions. This may be observed from the model’s persistent effort to maintain an outstanding level of performance, as illustrated in Table 2. The model demonstrated its flexibility and adaptation in the face of increasing rates of adverse poisoning. This was obvious from the relatively steady global accuracy metrics, even in situations where the integrity of client data was corrupted.

5.2. Significance of Data Reconstruction

The impact of adversarial attacks on the federated learning paradigm, specifically in relation to model accuracy and classification abilities, is unquestionable. This is demonstrated in Figure 10. Nevertheless, a major turn occurs with the implementation of data reconstruction approaches. The observed increase in the number of true positive results after the reconstruction process indicates that the implementation of strategic data restoration techniques has the potential to mitigate the harmful impacts of adversarial attacks. The robustness of the model, including its ability to recover from losses as shown in Figure 11, emphasizes the significant impact of effective data reconstruction methods.

5.3. Comparison with Previous Work

The findings of our study can be situated within the broader framework of the N-BaIoT dataset study, which focused on the detection of IoT botnet attacks [7]. The dataset was aimed to enable the distinction between benign and malicious network traffic. However, our research expanded its applicability by investigating its potential inside a federated learning framework with the novel IALM RPCA approach, particularly in the presence of adversarial circumstances. The expansion of the dataset’s application scope is not only enhanced, but it also highlights the practicality of employing federated learning models in real-world situations that may involve limitations.

5.4. Future Research Directions

Although our work provides a full overview of the federated learning landscape with momentum-based IALM RPCA in the presence of adversarial situations, there are still some areas that have not been explored:

• This study aims to conduct a comprehensive examination of advanced data reconstruction strategies, with the potential to enhance the recovery capabilities of compromised models to a greater extent.
• The expansion of this research to include other publically accessible datasets would serve to enhance the validation breadth, hence broadening the applicability of the findings.
• An investigation on the extent to which these findings can be scaled, particularly in cases where the number of clients (K) is significantly increased.
• This study is designed to investigate and evaluate advanced adversarial techniques and their corresponding responses in the context of federated learning.

In summary, our study highlights the undeniable potential of federated learning in addressing adversarial challenging situations. When coupled with effective data reconstruction techniques, these models have the ability to demonstrate durability, therefore guaranteeing their potential usability in various real-world situations where data integrity may be compromised. The findings, at their essence, highlight the inherent robustness of the federated learning paradigm, which is especially evident when the paradigm is further strengthened by capable data reconstruction approaches. Even while threatening activities can in fact affect performance, our research has shown that strategic countermeasures with novel approach can guarantee model robustness and capabilities.

6. Conclusions

In this paper, we present an in-depth investigation of the complexities of a multi-stage federated learning model, with an emphasis on the model’s defense against information integrity attacks in IoT contexts. The core focus of the study focuses on the integration of attack detection and data recovery, which constitutes a novel technique. This is demonstrated by the comprehensive examination of poisoning attacks, namely those arising from the presence of sparse noise inside the environment of the IoT. The results, which were emphasized by the utilization of the IALM RPCA algorithm, depict a promising outlook on the robustness of federated learning in handling reconstructed data following an attack. The results obtained from this research provide a significant basis for the evolving IoT environment and the increasing significance of decentralized learning approaches like federated learning. Moreover, a novel approach has been detected which outperforms traditional IALM RPCA in case of reconstruction. Nevertheless, the fact that this journey has been informative makes it evident that the continuously developing domain of IoT security and training needs further investigation, hence demonstrating the potential for ongoing advancements and enhancements in this domain.