This section assesses the proposed system using a benchmarking scenario and performance metrics described in the previous section. It also explains the environment setup for simulating the proposed system.
4.2. Centralized Learning vs. FL Based on Edge–Fog–Cloud Architecture
In this section, an analysis of the classification performance is conducted, comparing a traditional IDS with centralized learning and a CIDS with FL based on edge–fog–cloud for IoT. The performance evaluation encompasses metrics such as accuracy, precision, recall, and F1-score. Additionally, the loss parameter is examined to quantify the disparity between predicted values of the model and the actual ground-truth values in the dataset. Furthermore, parameters including training time, average CPU, and memory usage are analyzed for comparison in a lightweight analysis. This study delves into these parameters at each layer, starting from the edge layer, progressing to the fog layer, and culminating at the cloud layer where the global model is aggregated. It compares the proposed system with a centralized learning mode using the same dataset and DNN machine learning algorithm in terms of detection performance.
4.2.1. Centralized Learning Performance
The results of centralized learning are presented in
Table 5. The loss value indicates the error of the model’s predictions compared to the true labels. In this case, the loss value is 0.0045648, which suggests that the model’s predictions are relatively close to the true labels, indicating good performance. Accuracy measures the proportion of correctly classified samples out of the total number of samples. With an accuracy of 99.36%, the model correctly classifies a high percentage of the samples. Precision measures the proportion of true-positive predictions out of all positive predictions made by the model. With a precision of 99.55%, the model has a high precision, indicating that when it predicts a positive class, it is usually correct. Recall measures the proportion of true-positive predictions out of all actual positive samples in the dataset. With a recall of 99.79%, the model effectively captures a high percentage of the positive samples. The F1-score is the harmonic mean of precision and recall, providing a balance between the two metrics. With an F1-score of 99.67%, the model achieves a high balance between precision and recall, indicating robust performance. Additionally, the training time for the model was substantial, totaling 4041.07 s, suggesting comprehensive training and optimization processes. Despite the extensive training duration, the model’s resource utilization remained modest, with an average CPU usage of 4.7% and memory consumption of 40.78 GB.
4.2.2. FL on Edge Layer Performance
Now, we analyze the results from FL within the edge–fog–cloud architecture. In this analysis, each graph illustrates the progression of devices over multiple rounds (on the x axis), during which they compute the loss function value (on the y axis) associated with their segment of the model training. The initial analysis focuses on assessing the performance of the loss parameter. The performance outcomes of the loss parameter at the edge layer are depicted in
Figure 4. The graph comprises five line graphs labeled from A to E, each representing the ‘loss trend’ for a different ‘fog device’ (from Device 1 to Device 5) across several rounds of an FL process within an edge–fog–cloud architecture. Each ‘fog device’ oversees five ‘edge devices’ involved in the learning process, with the colors in the curves corresponding to different edge devices contributing to the learning process.
The loss function serves as a metric for evaluating the model’s predictive accuracy against the actual data, with lower values indicating superior performance. The graphs illustrate fluctuations in loss values across rounds, reflecting the iterative process towards improved performance. Among the fog devices, variations in average loss across their respective edge devices are observed. Notably, for Fog Device 1, Edge Device 1 stands out with the lowest average loss of 0.0067, indicating effective learning, while Edge Device 4 exhibits the highest average loss of 0.0102, suggesting potential challenges in model training or data quality. Fog Device 2 generally demonstrates lower average losses, with Edge Device 1 displaying the lowest average loss at 0.0064, though Edge Device 4 shows a slightly higher average loss of 0.0087, indicating varying levels of learning effectiveness. Fog Device 3 consistently maintains efficient learning across its edge devices, all exhibiting low average losses, with Edge Device 5 showcasing the lowest average loss of 0.0063, suggesting slightly better performance. On the other hand, Fog Device 4 displays significant variance in average loss among its edge devices, with Edge Device 2 recording the highest average loss of 0.0118, possibly indicating specific learning challenges or data processing issues, while Edge Device 3 demonstrates a notably lower average loss of 0.0066, implying more effective learning. Lastly, Fog Device 5 and Fog Device 3 demonstrate consistent performance with low average losses across edge devices, with Edge Device 4 leading with the lowest average loss of 0.0061, indicating superior learning efficiency.
This analysis indicates that while most edge devices perform consistently with low average losses, certain devices (e.g., Edge Device 4 from Fog Device 1 and Edge Device 2 from Fog Device 4) exhibit higher losses, which may highlight areas for potential improvement or investigation into the training data or model parameters. Overall, the FL system appears to perform well, with most devices demonstrating efficient learning capabilities.
The subsequent analysis focuses on the accuracy of the proposed system. The performance outcomes concerning the accuracy parameter at the edge layer are depicted in
Figure 5.
Among the edge devices under each fog device, variations in average accuracy are observed. In Fog Device 1, the edge devices display closely grouped average accuracy values, with Edge Device 1 achieving the highest average accuracy of 99.07%, while Edge Device 4 exhibits the lowest at 98.76%, indicating marginal performance differences. Similarly, Fog Device 2 demonstrates closely aligned performance across its edge devices, with Edge Device 1 leading slightly with an average accuracy of 99.09%, and Edge Device 4 recording the lowest at 98.88%. Notably, Fog Device 3 stands out for its exceptional and consistent performance across edge devices, with Edge Device 4 attaining the highest average accuracy of 99.12%, showcasing effective coordination. Conversely, Fog Device 4 presents a broader range of average accuracies among its edge devices, with Edge Device 3 performing well at 99.06%, while Edge Device 2 lags behind at 98.60%, indicating significant performance variance. Lastly, Fog Device 5 demonstrates robust and consistent accuracy across its edge devices, with Edge Device 4 leading with an average accuracy of 99.12%, reflecting effective learning and model optimization strategies. These insights suggest that while most edge devices across the fog devices perform consistently well, there are notable exceptions (e.g., Edge Device 4 from Fog Device 1 and Edge Device 2 from Fog Device 4) that highlight areas for potential improvement. Overall, the FL system shows a promising capacity for maintaining high accuracy levels across a diverse set of edge devices.
The subsequent analysis pertains to the precision of the proposed system. The performance outcomes related to the precision parameter at the edge layer are illustrated in
Figure 6.
Precision levels across edge devices under each fog device vary in their predictive accuracy. In Fog Device 1, the edge devices demonstrate high precision, with Edge Device 1 leading with an average precision of 99.68%, while Edge Device 4 exhibits the lowest average precision at 99.29%, still indicating a high level of predictive accuracy. Fog Device 2 maintains consistent high precision across its edge devices, with Edge Device 5 standing out with the highest average precision of 99.73%, showcasing exceptional performance in predicting positive classes. Similarly, Fog Device 3 also exhibits high precision across all edge devices, with Edge Device 5 slightly leading with an average precision of 99.72%, indicating very accurate model predictions. In contrast, Fog Device 4 shows more variance in precision among its edge devices, with Edge Device 2 having the lowest average precision of 98.98%, suggesting room for improvement, while Edge Device 3 displays a strong average precision of 99.67%. Fog Device 5, like the other fog devices, demonstrates high precision across its edge devices, with Edge Device 4 achieving the highest average precision of 99.68%, indicating very accurate identification of positive cases.
Overall, the FL system under study showcases impressive precision across most edge devices and fog devices, indicating a strong ability to accurately predict positive outcomes. The slight variances observed suggest targeted opportunities for model optimization, especially for the devices with relatively lower precision scores. This analysis underscores the system’s effectiveness in precision-oriented tasks, with specific areas identified for further enhancement.
The subsequent analysis pertains to the recall of the proposed system. The performance outcomes related to the recall parameter at the edge layer are depicted in
Figure 7.
The performance of edge devices under each fog device in terms of recall varies in their ability to identify true positives accurately. Fog Device 1 showcases high recall rates, with Edge Device 4 leading with the highest average recall of 99.45%. The differences among the devices are minimal, indicating consistent performance in detecting true positives. Fog Device 2 follows a similar trend with high recall rates across its edge devices. Edge Device 3 exhibits a slightly higher average recall of 99.43%, while Edge Device 5 shows the lowest at 99.24%, suggesting a small variance in minimizing false negatives. Fog Device 3 demonstrates a uniform performance in recall across its edge devices, with Edge Device 4 leading with an average recall of 99.39%, indicating a balanced detection rate of positive instances. Fog Device 4 presents a notable trend, with Edge Device 2 achieving the highest average recall of 99.60% and Edge Device 3 slightly lower at 99.36%, highlighting a robust ability to capture true positives, especially in Edge Device 2. Fog Device 5 maintains strong recall rates, with Edge Device 1 displaying the highest average recall of 99.50%. The performance across edge devices is closely matched, reflecting the effective identification of positive instances.
These insights suggest that the FL system, across most edge devices and fog devices, achieves a commendable level of recall, efficiently identifying true positives. The minor variances observed among some devices provide focal points for further optimization to enhance the system’s overall sensitivity. This analysis confirms the system’s effectiveness in scenarios where missing a positive detection is critical.
The next analysis regards the F1-score of the proposed system. The performance results of the F1-score parameter in the edge layer can be seen in
Figure 8.
The F1-scores of the edge devices under each fog device reflect their ability to strike a balance between precision and recall in model performance. Fog Device 1 demonstrates high and consistent F1-scores across its edge devices, with Edge Device 1 leading slightly with an average F1-score of 99.52%. The variance among the devices is minimal, indicating a stable performance in achieving the desired balance. Similarly, Fog Device 2 exhibits strong and consistent F1-scores, with Edge Device 1 having the highest average F1-score of 99.53% and Edge Device 4 showing the lowest at 99.43%, suggesting slight differences in model optimization. Fog Device 3 showcases exceptional balance, with Edge Device 4 achieving the highest average F1-score of 99.55% and all edge devices performing consistently well. In contrast, Fog Device 4 displays a wider range of average F1-scores, with Edge Device 3 scoring high at 99.52% and Edge Device 2 the lowest at 99.29%, indicating variability in balancing precision and recall effectively. Fog Device 5 maintains strong performance across its edge devices, with Edge Device 4 leading with the highest average F1-score of 99.55%. The performance consistency among the devices suggests a uniformly effective approach to minimizing both false positives and false negatives.
These findings suggest that the FL system, on average, achieves a commendable balance between precision and recall across most edge devices and fog devices. The slight variances observed among some devices provide opportunities for targeted improvements to further enhance the models’ effectiveness. Overall, the system demonstrates a strong capability in achieving high F1-scores, which is indicative of well-balanced and effective model performance.
4.2.3. FL on Fog Layer Performance
After analyzing the detection performance at the edge layer, we proceed to examine the detection performance at the fog layer. The results of the detection performance at the fog layer are illustrated in
Figure 9.
We begin by observing the loss values in the fog layers, which exhibit significant variations across fog devices, indicating a heterogeneous performance landscape. Subsequently, we delve into the analysis of accuracy performance in the proposed system. The analysis of fog devices’ performance across rounds unveils intriguing trends regarding accuracy. Initially, in the first round, each device demonstrates stable starting accuracies exceeding 90%. Fog Device 2 displays a unique pattern, commencing and concluding with high accuracy but encountering a notable drop to nearly 0% accuracy in round 4. This sharp decline suggests a potential failure or misconfiguration that was subsequently rectified, leading to a swift recovery in accuracy. The rapid rebound of Device 2 hints at a robust error correction or recalibration process employed to efficiently restore its performance levels. Similar patterns are observed in precision, recall, and F1-score metrics. Notably, there is a slight deviation in the precision pattern when Fog Device 3 achieves a recall value of 1.0 in round 4.
4.2.4. FL on Cloud Layer Performance
After analyzing the detection performance in the fog layer, we proceed to examine the detection performance in the cloud layer or global model. The results of the detection performance in the cloud layer or global model are illustrated in
Figure 10. The aggregation of the model in the cloud is also influenced by conditions in the fog layer. Notably, the accuracy of the global model drops to near 0% in round 4, while the other parameters remain relatively stable at over 90% in each round. After five rounds of training, the final results for accuracy, precision, recall, and F1-score consistently stand at 97.65%, 97.65%, 100%, and 98.81%, respectively.
4.2.5. FL Based on Edge–Fog–Cloud Architecture (Training Time)
This next analysis focuses on the length of training times across the FL system, as depicted in
Figure 11. Upon examining the training times across the fog and edge devices, notable variations and patterns emerge. Fog Device 1 exhibits a range in training times, with Edge Device 4 recording the highest average training time (159.00 s) and Edge Device 1 the lowest (150.22 s), suggesting efficiency discrepancies in the learning process. Fog Device 2, conversely, demonstrates relatively consistent training times, with Edge Device 5 exhibiting slightly lower averages (151.67 s), indicating a well-distributed computational load. Fog Device 3 showcases consistent training times among its devices, with Edge Device 4 having a marginally higher average (155.84 s), suggesting minor variations in processing efficiency. In contrast, Fog Device 4 highlights Edge Device 1 as having the highest average training time (158.64 s), potentially indicating more intricate computations or data processing requirements. Fog Device 5 illustrates closely aligned training times, with Edge Device 5 recording the highest average (157.03 s), reflecting a balanced workload distribution among the devices.
The analysis of training times across the fog and edge devices reveals intriguing insights into the efficiency and workload distribution within the system. While Fog Device 1 shows variability in training times, suggesting differing efficiencies in the learning process among its edge devices, Fog Device 2 demonstrates a more uniform distribution of computational load. Fog Device 3 maintains consistency in training times with minor variations, indicating stable processing efficiency. In contrast, Fog Device 4 exhibits disparities, with Edge Device 1 requiring more time, potentially due to complex computations. Fog Device 5 showcases balanced training times, implying an even workload distribution. These findings underscore the importance of optimizing computational resources and workload allocation to enhance overall system performance and efficiency.
4.2.6. FL Based on Edge–Fog–Cloud Architecture (Resource Consumption)
The next analysis is on CPU and memory consumption. Each fog devices coordinate five edge devices to undertake the learning process. The learning process is undertaken in five rounds in each edge device. The analysis explains the result by averaging CPU and memory consumption values in five rounds of the learning process from each edge device.
The CPU usage of the learning process in the FL process can be seen in
Figure 12. Examining the CPU usage of the learning process in edge devices provides valuable insights into the computational demands and workload distribution within the system. Fog Device 1 demonstrates relatively consistent CPU usage levels, with Edge Device 1 exhibiting slightly higher usage (4.87%). In contrast, Fog Device 2 presents a uniform CPU usage profile, with Edge Device 3 recording the highest average (4.88%). Fog Device 3 shows minimal variance in CPU usage. On the other hand, Fog Device 4 displays a range in CPU usage, with Edge Device 1 having the lowest usage (4.68%). Lastly, Fog Device 5 maintains relatively uniform CPU usage, with minor fluctuations implying a balanced utilization of computational resources across the devices. These observations emphasize the importance of optimizing CPU usage and workload distribution to enhance system performance and efficiency.
The analysis of CPU usage across fog and edge devices unveils crucial patterns regarding computational demands and workload distribution within the system. While Fog Device 1 and Fog Device 2 show consistent and uniform CPU usage profiles, respectively, indicating potential correlations with training efficiency and balanced computational demands, Fog Device 3 demonstrates an even distribution of computational workload. In contrast, the varying CPU usage in Fog Device 4 and the balanced usage in Fog Device 5 reflect potential implications on training times and resource utilization. These findings underscore the significance of optimizing CPU usage and workload allocation to maximize system performance and efficiency while ensuring a balanced distribution of computational resources across devices.
The next analysis highlights the memory usage across the FL system, as illustrated in
Figure 13. The examination of the memory usage of the FL process in each edge device reveals intriguing insights into memory utilization and workload distribution within the system. Across the edge devices coordinated by Fog Device 1, memory usage ranges from 29.706 GB (Edge Device 2) to 30.692 GB (Edge Device 1). For Fog Device 2, memory usage shows a broader range, from 31.668 GB (Edge Device 1) to 33.584 GB (Edge Device 4). Similarly, for Fog Device 3, memory usage is varied, with 30.878 GB (Edge Device 4) being the lowest and 32.708 GB (Edge Device 2) being the highest. Moving to Fog Device 4, memory usage increases progressively from 31.112 GB (Edge Device 1) to 33.600 GB (Edge Device 5). Lastly, Fog Device 5 exhibits the highest memory usage, with Edge Device 2 reaching up to 35.254 GB, indicating a significant memory load. The analysis of memory usage across the edge devices coordinated by different fog devices offers valuable insights into memory utilization patterns and workload distribution within the system. Interestingly, the examination reveals varying degrees of memory usage among the edge devices, with each fog device overseeing a distinct range of memory consumption. The variation in memory usage across fog devices shows the importance of efficient workload distribution and resource allocation strategies to optimize system performance and ensure balanced resource utilization. Additionally, the observed variations in memory usage highlight the complexity of managing resources in distributed computing environments and emphasize the need for tailored optimization techniques to enhance system scalability and efficiency.
Overall, while most fog devices demonstrate a balanced approach to memory utilization, certain edge devices exhibit higher average memory usage, indicating potential areas for optimization. Ensuring that memory usage is managed efficiently across all devices is crucial for maintaining the performance and scalability of the FL system, especially in resource-constrained environments. These findings may guide targeted improvements to memory management practices within the system.
4.2.7. Latency Performance
The next analysis compares network latency between edge–fog–cloud architecture and cloud-centric architecture. This comparison helps assess the performance of FL in both architectures. The data size of the training model in FL matches that of centralized learning, at 46 Kb, due to identical DNN architecture. However, the data size variable is not enough to estimate latency; we need to make some assumptions about the network’s transmission speed. Therefore, once the model data size is determined, simulating network transmission speed becomes necessary.
In networking, transmission speed refers to the rate at which data are transferred from one location to another. Transmission speed helps in understanding the consistency and stability of the network’s performance by indicating how long the network sustains a certain level of the actual rate at which data are being transmitted. Transmission speed is a metric used in networking to describe the duration for which a certain amount of the data transmission rate is available or utilized within a given time frame. It measures how long a particular data transmission rate level is sustained or maintained over a period of time. Understanding transmission speed is important for network administrators and engineers to ensure that network performance meets the requirements of applications and users. Transmission speed is also important to identify any issues or bottlenecks that may affect network performance over time [
36,
37]. This research used Formula (
13) to calculate transmission speed.
where
is the total size of packets in the flow and
is the duration of the flow of packets. This formula calculates the transmission speed using the flow duration as the time frame over which the total size of packets is transmitted. This equation gives the result of the average transmission speed for each flow [
38,
39].
In this study, the training model was executed over five rounds of training, providing a preliminary insight into FL dynamics. However, in practical scenarios, FL operates in a continuous fashion as edge devices, such as smartphones and IoT devices, perpetually generate new data. This continuous training process aligns with the dynamic nature of data production at the edge. To capture the realism of this ongoing learning paradigm, this study utilized transmission speed data derived from actual network traffic to simulate latency. By integrating traffic data from a comprehensive dataset comprising 140 million samples, this research aimed to extend the analysis beyond the initial five training rounds. The endeavor sought to meticulously model the training model data latency within the edge–fog–cloud architecture across the extensive dataset, ensuring a comprehensive exploration of FL dynamics under realistic conditions.
In this study, transmission speed measurement is conducted using data from the CICIoT2023 dataset. The dataset includes features such as ‘flow_duration’, which represents the duration of the packet’s flow, and ‘Tot size’, which represents the packet’s length. By utilizing these two features, this research calculates the transmission speed simulation using Formula (
13) for each traffic scenario. The simulation results of the transmission speed measurement using the CICIoT2023 dataset are illustrated in
Figure 14. The scenario compares communication latency among edge, fog, and cloud architectures with cloud-centric architectures, starting by taking the data from
Figure 14 and processing them in the simulation. This research sets the data size of the model at 46 kilobytes, likely representing the data size of the model to be transmitted. The simulation uses Formula (
10) to simulate latency for each instance of traffic in the CICIoT2023 dataset. The latency simulation scenario in this research can be seen in Algorithm 3.
Algorithm 3: Latency Simulation |
Data: CSV CICIoT2023 dataset Result: latency_edge_to_fog, latency_fog_to_cloud, and latency_edge_to_cloud Load the CSV dataset Calculate the average speed of data transfer for each task BFD ← df[’Tot size’] / df[’flow_duration’] Determine the size of a model model_size ← 46 KB Divide the devices into different groups num_edge_devices ← 5 num_fog_devices ← 5 num_total_edge_devices_cloud_centric ← 25 Assign data transfer speed to each group trnsSPD_edge_to_fog ← BFD / num_edge_devices trnsSPD_fog_to_cloud ← BFD / num_fog_devices trnsSPD_edge_to_cloud ← BFD / num_total_edge_devices_cloud_centric Calculate the time taken for data transfer between different groups latency_edge_to_fog ← model_size / trnsSPD_edge_to_fog latency_fog_to_cloud ← model_size / trnsSPD_fog_to_cloud latency_edge_to_cloud ← model_size / trnsSPD_edge_to_cloud |
The algorithm is designed to model the latency, or delay, involved in communication within an IoT system. It starts by ingesting a dataset named CSV CICIoT2023, likely containing information about IoT devices and their communication patterns. The goal of the algorithm is to calculate and provide three types of latencies: latency from edge to fog, latency from fog to cloud, and latency from edge to cloud (cloud-centric). To begin the process, the algorithm loads the dataset from the CSV file. It then calculates the average data transfer speed for each task, a crucial metric for understanding the speed at which data move between devices. This is achieved by dividing the total size of data transferred by the duration of the flow.
Next, the algorithm determines the size of a model, set at 46 Kb. This model size likely represents the amount of data processed or transferred by the IoT devices. The devices are categorized into different groups based on their roles or locations, with parameters indicating the number of edge devices, fog devices, and total edge devices in a cloud-centric architecture. Once the groups are established, the algorithm assigns data transfer speeds to each group. It computes the transmission speed for data transfer from edge devices to fog devices, from fog devices to the cloud, and from edge devices directly to the cloud, using previously calculated values and the number of devices in each group. Finally, the algorithm calculates the latency for data transfer between these groups. It determines the time taken for data transfer from edge devices to fog devices, from fog devices to the cloud, and from edge devices directly to the cloud using the model size and the transmission speed values for each group.
With the available transmission speed, the simulation proceeds to distribute it among the devices in the architecture. It calculates the transmission speed allocated for communication links from edge to fog, fog to cloud, and edge to cloud. Subsequently, the simulation computes the time taken for communication between different layers of the architecture. These calculations are based on the model size and the allocated transmission speed for each communication link. The plots visualize the communication latency for communication from edge to fog, fog to cloud, and edge to cloud, as seen in
Figure 15,
Figure 16, and
Figure 17, respectively.
The result from the cloud-centric architecture in
Figure 17 has bigger latency than edge to fog in
Figure 15 and fog to cloud in
Figure 16. The latency can be reduced by more than 50% when the fog–edge–cloud architecture is employed in the CIDS architecture. In edge–fog–cloud architecture, model updates are processed and aggregated locally at fog devices before being sent to the cloud server. Latency reduction is achieved by minimizing the distance data need to travel for processing and aggregation. Local aggregation at fog devices can reduce the latency associated with transmitting data to a centralized cloud server. In cloud-centric architecture, all model updates are sent directly to the centralized cloud server for aggregation. Latency is determined by the network speed between edge devices and the cloud server, as well as the processing time for model aggregation at the cloud server. Latency can be higher in cloud-centric architecture compared to edge–fog–cloud architecture due to the longer distance data need to travel to reach the cloud server and the potential network congestion.
In FL, a process utilized in distributed machine learning systems, we implement an averaging training model approach. This method is particularly relevant in the context of the edge–fog–cloud architecture, where computational resources are distributed across various tiers. Specifically, within this architecture, there exist 5 edge devices linked to each fog device, while 5 fog devices are connected to the cloud server, amounting to a total of 5 fog devices and 25 edge devices. Upon completion of the training process on each edge device, a training model with a data size of 46 Kb is generated. Subsequently, these trained models are transmitted to the fog devices. Here, an averaging process takes place, wherein the training models received from the edge devices are combined to produce a unified training model, still maintaining a data size of 46 Kb. This averaging mechanism significantly reduces the amount of data that needs to be transmitted to the cloud. With the presence of fog devices, only 46 Kb of data needs to be sent to the cloud from five fog devices. However, in the absence of fog devices, all 25 edge devices would have to directly transmit their training models to the cloud server. This illustrates the efficiency gained by utilizing fog devices as intermediate computational processors in the edge–fog–cloud architecture, as it minimizes the burden on the cloud server by reducing the amount of data transmission required directly to it.
4.2.8. Performance Comparison
Based on the analysis conducted for FL and centralized learning, a comparative examination provides valuable insights across various parameters. Regarding loss, the FL approach displays fluctuations in loss values across rounds and devices, indicating an iterative process toward enhancing model performance. In contrast, the centralized learning model consistently maintains low loss values, suggesting robust predictive capabilities. Across fog devices and their respective edge devices, accuracy, precision, recall, and F1-score metrics demonstrate consistent and efficient learning, albeit with slight variations among devices. Meanwhile, in centralized learning, these metrics notably remain high, all surpassing 99%, indicating a strong and uniform detection performance across the entire dataset.
The accuracy, precision, recall, and F1-score parameters measured in percentages reflect high-performance levels in centralized learning. Specifically, accuracy, precision, recall, and F1-score values reach 99.36%, 99.55%, 99.79%, and 99.67%, respectively. In contrast, the parameters obtained from FL, utilizing an edge–fog–cloud architecture after five rounds of training, exhibit slightly lower performance levels compared to centralized learning. These include accuracy, precision, recall, and F1-score values at 97.65%, 97.65%, 100%, and 98.81%, respectively.
In centralized learning, training time tends to be higher, averaging 4041.07 s, as data processing occurs sequentially on a single server. This sequential processing may result in longer training times, especially with large datasets. However, in FL, the average training time across edge devices is significantly lower at 155 s. This reduction is attributed to distributed learning across edge devices, allowing for parallel processing capabilities. Simultaneous model training on multiple devices leads to faster convergence and ultimately reduces the overall training time.
The average CPU usage across edge devices in both FL and centralized learning remains consistent, with an average value of 4.7%. However, a notable difference emerges in memory consumption between the two approaches. In centralized learning, the average memory usage is notably higher, at 40.78 GB, compared to the FL scenario, where the average memory usage across edge devices is 32.15 GB. FL demonstrates efficient CPU usage, distributing computational demands across edge devices, thereby optimizing CPU resources and enhancing system performance. Conversely, centralized learning may experience CPU spikes due to the sequential processing of data on a single server, potentially leading to performance bottlenecks. Additionally, FL typically incurs lower memory consumption compared to centralized learning. The distributed data storage across edge devices ensures efficient memory utilization, with each device only requiring resources for processing its local data portion. In contrast, centralized learning’s centralized data storage necessitates larger memory capacity on the server, resulting in higher memory consumption compared to FL.
The edge–fog–cloud architecture proved to reduce latency compared to cloud-centric architecture by performing local aggregation at fog devices. High volumes of data being transmitted to and from the cloud server can lead to network bottlenecks in cloud-centric architecture, increasing latency and potentially affecting the overall training process. Therefore, this research proposed model aggregation performed at the fog nodes to reduce the need to send data to a centralized cloud server. By aggregating models in the fog layer, latency is further reduced compared to cloud-centric architectures. The proximity of edge and fog nodes to the devices minimizes the distance data need to travel, resulting in lower latency for model aggregation and updating.
4.4. CIDS Trust and Privacy-Preserving Analysis
In the CIDS, implemented within an edge–fog–cloud architecture and utilizing blockchain with a proof-of-work (PoW) consensus mechanism, preventing Sybil attacks is crucial for maintaining network integrity and trustworthiness. Each participating edge device and fog node maintains a copy of the blockchain ledger, serving as a decentralized and immutable record of all transactions, including model updates and node interactions. The PoW consensus mechanism ensures that transactions are validated and added to the blockchain through computationally intensive mining processes, making it challenging for malicious entities to tamper with the ledger.
To model the prevention of Sybil attacks mathematically, we denote variables such as the total number of nodes (N), total number of miners (M), total computational power of miners (P), target difficulty for PoW mining (T), current blockchain height (H), block reward (B), and cost of performing a Sybil attack (C). The probability of a Sybil attack succeeding () can be represented as . To prevent Sybil attacks, the computational power of legitimate nodes participating in PoW mining () must exceed the computational power of potential attackers (C). Thus, by ensuring that P exceeds C, the CIDS network mitigates the risk of Sybil attacks, safeguarding its trust and privacy-preserving capabilities.
By integrating blockchain with a PoW consensus mechanism into the CIDS network, the system ensures that Sybil attacks are economically infeasible for potential attackers. The computational resources required to perform a successful Sybil attack would surpass the computational power of honest nodes participating in PoW mining, thereby safeguarding the integrity and trustworthiness of the CIDS network.
In the CIDS, FL plays a pivotal role in preserving privacy by ensuring decentralized storage of sensitive data, while blockchain technology is utilized to uphold trust and transparency within the system. This combination of mechanisms is well suited to prevent data breaches and uphold privacy standards. In the CIDS ecosystem, each edge device trains its local intrusion detection model using locally collected sensitive network traffic data. These data remain decentralized and are never shared with other devices or the central server. Let N represent the total number of participating edge devices in FL, M denote the total number of fog nodes, and T represent the total number of transactions recorded on the blockchain. Furthermore, signifies the sensitive data stored on edge device i, denotes the local model trained on edge device i, and represents the model update transmitted from edge device i to fog node j.
After local training, edge devices transmit only model updates (not raw data) to the fog nodes for aggregation. This transmission process employs privacy-preserving techniques such as secure aggregation or differential privacy to maintain the confidentiality of individual contributions. These methods ensure that the aggregated model update does not divulge any sensitive information regarding the individual data of edge devices.
After the model updates are aggregated at the fog nodes, denoted as , they are bundled into a transaction and subsequently recorded on the blockchain. Acting as a decentralized ledger, the blockchain meticulously records all transactions pertaining to model updates and interactions between edge devices and fog nodes. This blockchain-based architecture ensures transparency and immutability, empowering all participants to verify the integrity of the recorded transactions. Through the fusion of FL for privacy-preserving model training and blockchain for transparent transaction recording, the CIDS adeptly averts data breaches and upholds trust within the system.
The CIDS leverages a sophisticated blend of FL and blockchain technology within an edge–fog–cloud architecture to fortify its trustworthiness and preserve user privacy. FL ensures that sensitive data remain decentralized, as each edge device trains its local intrusion detection model without sharing raw data. Model updates are aggregated at fog nodes using privacy-preserving techniques such as secure aggregation or differential privacy, safeguarding individual contributions. These aggregated updates are then recorded on the blockchain, serving as a transparent and immutable ledger that meticulously records all interactions and transactions within the system. This innovative fusion of FL and blockchain technology not only prevents data breaches but also instills confidence in the integrity and transparency of the CIDS ecosystem, fostering trust among its participants.