Next Article in Journal
A Label-Efficient Adaptive Retraining Framework for Network Intrusion Detection Under Streaming Arrival Bias
Previous Article in Journal
WACT: Lossless Compression of INT8 Weights and Activations to Reduce Data Traffic in Edge NPU Memory Hierarchies
Previous Article in Special Issue
IBPS—A Novel Integrated Battery Protection System Based on Novel High-Precision Pressure Sensing
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Three-Stage Federated Distillation Framework for Robust Intrusion Detection in Heterogeneous IoT/Edge Networks

1
School of Cybersecurity, Xi’an Polytechnic University, Xi’an 710048, China
2
Economic and Technological Research Institute of State Grid Henan Electric Power Company, Zhengzhou 450052, China
3
College of Economics and Management, Taiyuan University of Technology, Taiyuan 030024, China
*
Authors to whom correspondence should be addressed.
Electronics 2026, 15(17), 3810; https://doi.org/10.3390/electronics15173810
Submission received: 23 June 2026 / Revised: 23 July 2026 / Accepted: 28 July 2026 / Published: 25 August 2026
(This article belongs to the Special Issue IoT Sensing and Generalization)

Abstract

Internet of Things(IoT)/edge intrusion-detection systems operate on distributed traffic and system-state data whose distributions vary across gateways, services, and attack conditions. We study a server-assisted federated setting in which a teacher reference is fitted from a permitted server-accessible training pool and explicitly distinguish this simulation assumption from fully decentralized deployment. The proposed framework evaluates progressive local training through boundary stabilization, confidence-weighted decision distillation, representation alignment, and validation-quality-aware aggregation. The evaluation uses a leakage-controlled protocol: server and client validation subsets are held out before federated training, update quality and early stopping use validation data only, and the final-test split is evaluated once. Results on NSL-KDD, CIC-IDS2017, Edge-IIoTset, and the ToN-IoT network dataset show competitive primary performance and stronger robustness in several severe label-skew settings. On the Telemetry of Things(ToN-IoT) with Dirichlet alpha = 0.1, the proposed method achieves 91.46 ± 5.54 F1, compared with 53.73 ± 49.00 for FedAvg and 53.77 ± 48.92 for FedProx. The results do not establish universal superiority or a universally optimal stage order: competing methods remain stronger in selected stable and attack-shift settings. The framework is therefore presented as a bounded, server-assisted robustness-oriented training strategy for heterogeneous IoT/edge intrusion detection.

1. Introduction

Internet of Things (IoT) and edge networks are increasingly deployed in smart cities, industrial monitoring, healthcare, transportation, and autonomous systems. These environments are usually composed of distributed sensing devices, edge gateways, communication links, and cloud-assisted services. While such infrastructures improve real-time perception and decision making, they also generate large volumes of heterogeneous traffic and system-state data. Abnormal behaviors caused by cyberattacks, device faults, misconfigurations, or malicious access may propagate through edge gateways and affect the reliability of the entire system. Therefore, robust anomaly detection has become a fundamental component of secure IoT and edge intelligence [1,2,3,4,5,6,7].
However, anomaly detection in distributed IoT/edge networks faces a significant non-IID robustness challenge. Data collected by different gateways or device groups are often non-independent and identically distributed because they may differ in device type, protocol configuration, user behavior, deployment environment, traffic pattern, and anomaly ratio. A detector trained on one edge client may perform well locally but become unstable when aggregated with clients that observe different feature statistics or attack distributions. This problem is particularly serious when anomalous samples are rare because local models can easily learn biased decision boundaries dominated by majority normal traffic. As a result, this paper focuses on robustness under heterogeneous edge-client data rather than claiming broad real-domain transfer from label-skew simulations alone [2,3,8,9].
Federated learning (FL) provides a promising solution for distributed anomaly detection because it enables multiple edge clients to collaboratively train a global model without sharing raw traffic or system-state data. Classical methods such as FedAvg reduce communication cost by averaging local model parameters, while FedProx improves optimization stability by constraining local model drift [8,10]. Nevertheless, standard parameter-level aggregation is insufficient when client distributions are strongly heterogeneous. Different edge clients may produce inconsistent gradients, biased local classifiers, and unstable feature representations. In such cases, simple aggregation may amplify unreliable updates and degrade the global detector, especially under severe non-IID partitions [9,11,12].
Knowledge distillation offers an additional way to transfer decision and representation information across heterogeneous clients. Teacher soft labels can provide decision-level class-relation information, and feature-level distillation can encourage intermediate representations to become more consistent across clients [13,14,15]. These properties are useful for federated IoT intrusion detection, where hard labels and local empirical risks may be highly biased [16,17,18,19]. However, directly applying distillation throughout the whole local training process may introduce optimization conflicts. If strong distillation constraints are imposed too early, the student model may not establish a stable local decision boundary. If representation alignment is applied too late, client-specific bias may already have been embedded into the model. Therefore, a key question is how to schedule bias suppression, decision transfer, and representation alignment in a progressive and stable manner.
To address these issues, this paper studies a three-stage server-assisted federated distillation framework for robust intrusion detection in heterogeneous IoT/edge security networks. Rather than treating all distillation losses as a single uniform objective, the framework evaluates a progressive local-training protocol that separates boundary stabilization, teacher-guided decision transfer, and representation alignment. The substantive contribution is not a claim that these ingredients are individually new, nor that one fixed order is universally optimal. Instead, the contribution is an experimentally specified protocol that makes their timing, validation-based quality weighting, teacher assumption, and failure boundaries explicit under the same backbone and client partitions as direct FL, prototype, contrastive, and distillation baselines.
The evaluation uses NSL-KDD, CIC-IDS2017, Edge-IIoTset, and ToN-IoT. It covers balanced partitions, Dirichlet label skew, no-teacher and budget-matched controls, aggregation and schedule controls, and leave-one-attack testing. All results follow a leakage-controlled selection protocol with held-out server and local validation partitions. The empirical aim is deliberately bounded: to identify conditions under which the proposed server-assisted protocol is competitive or more stable, not to claim uniform superiority across data distributions or attack types.
The main contributions of this paper are summarized as follows:
(1)
We formulate and evaluate a progressive within-round training protocol that makes the timing of boundary stabilization, decision distillation, and representation alignment explicit. The contribution is a reproducible protocol-level comparison against joint and alternative schedules, rather than a claim that any individual loss term is new or that the default order is universally optimal.
(2)
We specify a server-assisted teacher mechanism that transfers confidence-weighted soft targets and intermediate features, while explicitly disclosing its data-access assumption. The teacher is fitted from a server-accessible pool constructed from client-training subsets in the experimental simulation; the paper therefore distinguishes this setting from fully decentralized federated learning and reports no-teacher and budget-matched controls.
(3)
We introduce validation-quality-aware aggregation based on post-training F1 from held-out local validation subsets. Final-test data are excluded from quality scoring, early stopping, checkpoint selection, and teacher fitting. This design directly tests whether update reliability can complement sample-size weighting under severe label skew.
(4)
We provide a leakage-controlled robustness evaluation across NSL-KDD, CIC-IDS2017, Edge-IIoTset, and ToN-IoT, including FedAvg, FedProx, FedBN, FedAdam, SCAFFOLD, MOON, FedProto, FLKD, and an FD-IDS-style surrogate. Five-seed severe-skew controls, schedule and aggregation analyses, server-assistance controls, and attack-type holdouts support a bounded conclusion: the method is competitive in stable settings and can be more robust in selected strong-heterogeneity settings, while several baselines remain preferable in other conditions.
The remainder of this paper is organized as follows. Section 2 reviews related work on IoT/edge anomaly detection, federated learning under non-IID data, and distillation-based representation alignment. Section 3 presents the proposed three-stage federated distillation framework. Section 4 reports the experimental results, ablation studies, and efficiency analysis. Section 5 discusses the findings, limitations, and deployment implications. Section 6 concludes the paper.

2. Related Work

2.1. Anomaly Detection in IoT/Edge Security Networks

IoT and edge networks have become important infrastructures for smart cities, industrial monitoring, healthcare, transportation, and autonomous systems. These networks usually consist of distributed sensing devices, edge gateways, communication links, and cloud-assisted services. During operation, they continuously generate network traffic, protocol-level records, and system-state features. Abnormal behaviors caused by cyberattacks, device faults, misconfigurations, or malicious access may affect the reliability and security of the entire IoT/edge system. Therefore, anomaly detection has become a key task for secure edge intelligence [1,2,3].
Traditional intrusion-detection and anomaly-detection methods can achieve promising performances when training and testing data follow similar distributions. However, real IoT/edge environments are usually heterogeneous. Different gateways may differ in device type, communication protocol, traffic pattern, deployment environment, and anomaly ratio. As a result, the detector trained in one domain may not generalize well to another domain. This issue is particularly important in security-oriented IoT scenarios because anomalous samples are usually rare and class distributions are highly imbalanced. Under such conditions, local models may easily form biased decision boundaries dominated by majority normal traffic [2,3,20,21].
Existing IoT security datasets, such as NSL-KDD, CIC-IDS2017, Edge-IIoTset, BoT-IoT, ToN-IoT, and N-BaIoT, provide useful benchmarks for evaluating anomaly detection under different traffic and attack conditions [4,5,6,7,22,23]. Among them, Edge-IIoTset is especially relevant to IoT/IIoT edge-security research because it contains heterogeneous sensors, protocols, edge/cloud configurations, and multiple attack categories [4]. However, these benchmarks also reveal a practical challenge: anomaly detectors should not only fit a single local data distribution but also remain robust when deployed across heterogeneous edge domains. This motivates the use of federated and robustness-oriented learning strategies for distributed IoT/edge anomaly detection.

2.2. Federated Learning Under Non-IID Edge Data

Federated learning provides a natural solution for distributed IoT/edge anomaly detection because it enables multiple clients to collaboratively train a global model without sharing raw data. FedAvg is the most widely used baseline, which reduces communication cost by averaging local model parameters [10]. FedProx further improves optimization stability by adding a proximal regularization term to restrict local models from drifting too far away from the global model [8]. These methods are simple and effective under moderately heterogeneous settings, and they have been widely used as basic baselines in federated intrusion-detection studies [1,16,24,25,26].
Nevertheless, standard federated optimization does not fully solve the non-IID problem. In IoT/edge security networks, different clients may observe different normal behaviors, attack categories, feature distributions, and class proportions. These differences can cause client drift, inconsistent gradients, and unstable global aggregation. FedBN shows that local normalization statistics can be an important source of heterogeneity and proposes to keep batch-normalization parameters local [9]. SCAFFOLD reduces client drift through control variates, and adaptive federated optimization methods improve server-side update stability [11,12]. These studies demonstrate that non-IID learning is a central problem in federated systems.
However, most parameter-stabilization methods mainly focus on optimization consistency rather than decision-level and representation-level generalization. When local clients have different anomaly ratios or traffic patterns, their classifiers may learn domain-specific decision boundaries. Directly averaging such models may weaken cross-domain robustness. Therefore, heterogeneous IoT/edge anomaly detection requires mechanisms that can coordinate not only model parameters but also decision distributions and intermediate representations.

2.3. Federated Distillation and Representation Alignment

Knowledge distillation transfers knowledge from a teacher model to a student model by using soft labels, intermediate representations, or relational information [13,14,15]. Compared with hard labels, teacher soft labels can provide richer class-relation information and reduce overfitting to local biased labels. Feature-level distillation can further encourage different models to learn more consistent intermediate representations. These properties make distillation attractive for federated learning under heterogeneous edge data.
Recent federated distillation studies have explored different forms of knowledge transfer, including output-level distillation, ensemble distillation, prototype-based transfer, contrastive representation learning, and model-contrastive learning [16,17,18,19]. These studies motivate the need to compare the proposed method not only with FedAvg and FedProx but also with non-IID-oriented baselines such as FedBN, SCAFFOLD, adaptive federated optimization, and MOON. In intrusion-detection scenarios, such baselines test different failure modes: feature-statistics shift, client drift, server-side optimization instability, and representation inconsistency.
Despite these advantages, existing distillation-based federated methods still face two limitations. First, many methods apply distillation as a uniform loss throughout local training. This may be suboptimal under strong non-IID conditions because early local training often suffers from unstable decision boundaries and local class imbalance. If strong distillation is imposed too early, the student model may receive unreliable guidance before forming a stable local representation. Second, output-level distillation alone may not sufficiently handle feature-distribution mismatch among heterogeneous edge domains. Therefore, a more progressive mechanism is needed to determine when to stabilize decision boundaries, when to transfer teacher decisions, and when to align intermediate representations.

2.4. Quality-Aware Aggregation in Heterogeneous Federated Learning

Another important issue in heterogeneous federated learning is how to aggregate local updates. Standard FedAvg uses sample-size-based weighting, assuming that clients with more data should contribute more to the global model [10]. However, in IoT/edge anomaly detection, a client with more samples is not necessarily more reliable. For example, a gateway may contain highly imbalanced data, noisy labels, domain-specific traffic, or unstable anomaly distributions. In such cases, sample-size-based aggregation may amplify low-quality local updates.
Several studies have attempted to improve server-side aggregation by considering optimization stability, adaptive learning rates, or client drift correction [11,12]. However, fewer works explicitly consider the quality of local anomaly-detection updates under strong non-IID security data. In anomaly detection, local performance indicators such as F1-score are useful because they jointly reflect precision and recall, especially when anomalous samples are imbalanced. Therefore, quality-aware aggregation can reduce the influence of unreliable clients and improve global robustness.
In this study, quality-aware aggregation is introduced to complement staged distillation. Instead of relying only on the number of local samples, the server assigns aggregation weights according to local update quality. This design is particularly suitable for heterogeneous IoT/edge security networks, where different clients may have significantly different anomaly distributions and training stability.

2.5. Research Gap and Motivation

Existing studies provide important foundations for federated IoT/edge anomaly detection, but several gaps remain. First, standard FL methods such as FedAvg and FedProx mainly stabilize parameter aggregation, but they do not explicitly address biased decision boundaries caused by local class imbalance and domain shift. Second, existing FL-KD methods often use a single-stage or uniform distillation strategy, which may introduce optimization conflicts when local decision boundaries are still unstable. Third, output-level distillation alone is insufficient to reduce representation mismatch among heterogeneous edge clients. Fourth, conventional sample-size-based aggregation may be unreliable when some clients produce unstable or biased updates.
To address these limitations, this paper proposes a three-stage federated distillation framework for robust anomaly detection in heterogeneous IoT/edge security networks. The framework progressively organizes local training into boundary stabilization, teacher-guided decision distillation, and representation alignment. This design first suppresses local boundary bias, then transfers decision-level teacher knowledge, and finally aligns intermediate representations after the local model becomes more stable. In addition, a quality-aware aggregation rule is used to reduce the influence of unreliable local updates. Compared with existing FL-KD intrusion-detection methods, the proposed framework emphasizes three underexplored aspects at the same time: the timing of distillation within a communication round, the hierarchy of decision- and feature-level transfer, and the reliability of client aggregation under strong non-IID updates. The stage order is an empirically motivated protocol whose role is assessed through the schedule-control experiments; it is not derived from a convergence theorem and is not claimed to be theoretically optimal.

3. Materials and Methods

3.1. Problem Formulation

Consider a federated IoT/edge intrusion-detection network with K edge clients, where each client represents an edge client group, device group, or edge gateway. Client k owns a local dataset D k = { x k , i y k , i | i = 1,2 , , n k } , where x k , i denotes the i-th traffic or system-state feature vector, y k , i is the corresponding binary label, and n k is the number of local samples. The total number of training samples is N = k = 1 K n k . Let θ t denote the global model maintained by the server at round t and θ k t denote the local model on client k . Standard FL minimizes a weighted sum of local empirical risks.
In this study, the target IoT/edge network context specifically refers to security-oriented network traffic records and system-state monitoring vectors, distinguishing it from physical-layer wireless sensing signals such as Wi-Fi CSI, RFID, UWB, or radar.
m i n θ F θ = k = 1 K n k N F k θ .
The local empirical risk on client k is defined as
F k θ = 1 n k i = 1 n k f θ x k , i , y k , i ,
where f θ (·) denotes the anomaly detector and ℓ(·) is the supervised classification loss. In heterogeneous IoT/edge intrusion-detection networks, local clients may differ in device configuration, protocol usage, traffic behavior, and anomaly distribution. Directly optimizing Equation (1) can therefore suffer from client drift and unstable non-IID aggregation. The proposed framework keeps the same FL interface but replaces the single local objective with three progressive stages designed for robustness-oriented training.

3.2. Base Detector and Preprocessing

The base anomaly detector is implemented as a multilayer perceptron. For a preprocessed input feature vector x R d , the student model and the global model use three fully connected feature blocks followed by a binary classification layer. The overall network structure is d 512 256 128 2 . Each feature block consists of a linear layer, batch normalization, ReLU activation, and dropout with a dropout rate of 0.2. The final classifier maps the 128-dimensional representation to two logits corresponding to normal and anomalous traffic. Therefore, the task is formulated as binary network anomaly detection rather than fine-grained multiclass attack classification.
The intermediate representations extracted from the three feature blocks are denoted as h 1 R 512 , h 2 R 256 , and h 3 R 128 . These hierarchical representations are used for feature-level distillation and representation alignment in the third training stage.
The experiments use NSL-KDD, CIC-IDS2017, Edge-IIoTset, and ToN-IoT as security-oriented intrusion-detection benchmarks. They evaluate network or edge-gateway traffic and system-state records, not physical-layer wireless sensing signals such as Wi-Fi CSI, UWB, RFID, or millimeter-wave radar. The ToN-IoT network telemetry data add a newer IoT/IIoT-oriented benchmark with heterogeneous network services and attack traffic. For ToN-IoT, high-cardinality metadata and direct identifiers are removed before binary anomaly detection, and the final train–test split is stratified before any server or client validation split is created.

3.3. Overview of the Proposed Framework

The framework contains an edge-side detector, a server-assisted teacher reference, and a validation-quality-aware aggregation module. In a fully decentralized deployment, clients can retain raw traffic locally and exchange models and scores. However, the present experiments additionally assume that the server can access a permitted training pool to fit the teacher reference. We therefore describe the method as server-assisted rather than claiming that teacher training is fully decentralized. This assumption is isolated through no-teacher and budget-matched controls.
Figure 1 illustrates the three-stage client update and server aggregation flow. In the experimental protocol, the teacher reference is fitted from a server-accessible pool derived from client-training subsets after server and client validation subsets are held out. This server-assistance assumption is explicit and is not presented as a fully decentralized deployment mechanism.
Figure 2 shows one communication round. The order 1→2→3 is the default protocol used for the main comparison, but alternative schedules and a joint objective are evaluated explicitly. The results are interpreted as schedule-dependent rather than as evidence that a single order must dominate across all datasets. To avoid ambiguity in the subsequent analysis, Table 1 consolidates the canonical terminology, intended role, and empirical evidence associated with each component of the proposed framework.

3.4. Three-Stage Local Training

3.4.1. Stage 1: Boundary Stabilization with Bias Suppression

The first stage aims to stabilize the initial decision boundary under local class imbalance and client-specific traffic behavior. Instead of freezing the feature extractor, the current implementation updates all parameters with mild optimization strength and a bias-suppression regularizer. The Stage 1 objective is
L 1 = L c + λ 1 L F B R
where L c is the class-weighted cross-entropy loss with label smoothing, L FBR is the forward bias suppression regularization term, and λ 1 controls its strength. Let z S , i be the student logits for the i-th sample in a mini-batch of size B . The average predicted class probability of the student model is
p ¯ S = 1 B i = 1 B s o f t m a x z S , i .
Let u denote the uniform class distribution, u = (1/C, 1/C, …, 1/C), where C is the number of classes. In this work, C = 2. The forward bias suppression term is defined as
L FBR = K L p ¯ S u .
By penalizing excessive deviation from a balanced prediction distribution at the early stage, this term helps reduce the influence of local class imbalance on the initial decision boundary.

3.4.2. Stage 2: Teacher-Guided Decision Distillation

The second stage performs full-parameter fine-tuning and introduces teacher-guided soft-label distillation. In heterogeneous IoT/edge intrusion-detection clients, the teacher provides a more stable decision reference than purely local hard labels. The Stage 2 objective is
L 2 = L c + λ 2 L IFBR + λ 3 L KD ,
where L IFBR is the inverse bias suppression term, L KD is the knowledge distillation loss, and λ 2 and λ 3 are their corresponding weights. The inverse bias suppression term is defined as
L IFBR = K L u p ¯ S .
Compared with Stage 1, this inverse form encourages the model to maintain sufficient coverage of both classes during full-parameter fine-tuning, thereby reducing excessive shrinkage toward local majority classes. Let z T and z S denote the teacher and student logits, respectively. With distillation temperature τ , the softened teacher and student distributions are
p T τ = s o f t m a x z T τ , p S τ = s o f t m a x z S τ .
The decision-level distillation loss is
L KD = τ 2 K L p T τ p S τ .
In implementation, the effective weight of L KD is adjusted according to teacher confidence. When the teacher prediction is more confident and stable, the distillation strength is increased; otherwise, it is reduced to alleviate possible negative transfer from unreliable soft labels.

3.4.3. Stage 3: Representation Alignment and Feature Calibration

The third stage extends distillation from the output space to the representation space. This stage encourages heterogeneous clients to learn more consistent intermediate representations, thereby improving robustness under non-IID training. The Stage 3 objective is
L 3 = L c + λ 4 L MKD + λ 5 L FLD + λ 6 L cal + L prox ,
where L MKD is the cross-module distillation loss, L FLD is the feature-level distillation loss, L cal is the feature distribution calibration loss, and L prox is an optional proximal regularization term. The cross-module distillation loss adopts a symmetric KL divergence between the softened teacher and student output distributions:
L MKD = 0.5 K L p T τ p S τ + 0.5 K L p S τ p T τ .
The feature-level distillation loss aligns the intermediate representations of the student and teacher models:
L FLD = l = 1 L h S l h T l 2 2 .
Here, h S l and h T l denote the student and teacher features at layer l , respectively, and L is the number of feature layers used for distillation. In this work, L = 3 , corresponding to the 512-, 256-, and 128-dimensional feature blocks. The feature distribution calibration term is applied to the first feature block:
L cal = μ h S 1 μ h T 1 2 2 + σ h S 1 σ h T 1 2 2 ,
where μ(·) and σ(·) denote the mini-batch mean and standard deviation of the feature representation. This term encourages the first-level feature distribution to remain stable and reduces local feature drift caused by heterogeneous client data. The proximal regularization term is defined as
L prox = μ p 2 θ k θ t 2 2 ,
where μ p is the proximal coefficient, θ k is the local model parameter, and θ t is the global model parameter broadcast by the server at round t . This term limits the deviation in local updates from the global model and improves training stability under strong heterogeneity.

3.5. Server-Side Teacher and Quality-Aware Aggregation

The teacher model is maintained on the server side. The implementation first fixes the final train–test split, then holds out 10% of the training split for server-side model selection and holds out 10% of each client subset for local validation. The teacher pool is the union of the remaining client-training subsets. Thus, the pool contains no final-test samples and no held-out validation samples, but it is server-accessible in this simulation. This is a material deployment assumption, not a privacy claim. The pool size, class composition, split indices, random seeds, and hyperparameters are recorded in the experiment artifacts.
To separate the contribution of server assistance from the training protocol, this study reports three explicit controls: (i) an Ours w/o teacher variant, (ii) an Ours budget-matched variant with comparable local optimization budget, and (iii) direct FLKD and FD-IDS-style distillation baselines. These controls do not eliminate the practical limitation of requiring server-accessible teacher data. Accordingly, Section 5 states that the proposed method is not directly applicable to a fully decentralized setting without an alternative public, synthetic, or privacy-preserving teacher source.
After local training, each client uploads model parameters and a scalar quality score. The score is calculated from the post-training F1 on that client’s held-out local validation subset and normalized across the selected clients. It is never calculated on the final-test set. Server validation F1 is used for early stopping and checkpoint selection; final-test performance is calculated once after the selected checkpoint is restored.
a k t = q k t + ε j = 1 K q j t + ε .
The global model is updated by
θ t + 1 = k = 1 K a k t θ k t + 1 .
Compared with sample-size-only aggregation, this strategy emphasizes the reliability of the current local update rather than simply reflecting the client data scale. Therefore, it can reduce the influence of unstable or low-quality client updates under strong non-IID conditions. For teacher updating, an optional exponential moving average strategy can be used:
θ T t + 1 = 1 β θ T t + β θ t + 1 .
The teacher-update option is retained as an implementation detail. The primary clean-protocol experiments use the fixed server-assisted training pool; optional exponential moving-average updates are not used to support the central conclusions of this study.

3.6. Computational Cost

Compared with FedAvg and FedProx, the proposed framework introduces additional training costs from three sources. First, Stage 2 and Stage 3 require additional distillation loss computation. Second, the server maintains and updates an auxiliary teacher model. Third, quality-aware aggregation requires each client to compute a local post-training quality score. However, the proposed method does not increase the number of communication rounds. Moreover, the teacher model is only used during training and is not required during inference. Therefore, the inference-time model structure remains the same as the student detector, and no additional inference parameters are introduced.
Let P denote the number of student model parameters and R denote the maximum number of communication rounds. The parameter upload cost of each client per round remains approximately O P . If the teacher model is separately downloaded by clients, the downlink communication may increase by another model parameter size, also approximately O P . Therefore, the proposed method should be understood as a training-stage enhancement framework. Its additional cost is mainly reflected in local computation and possible teacher-model downlink transmission, rather than in inference-time deployment complexity.

4. Results

4.1. Experimental Setup

The evaluation uses four public dataset sources: NSL-KDD, CIC-IDS2017, Edge-IIoTset, and ToN-IoT. NSL-KDD is evaluated under two protocols: the official split includes the train–test shift, whereas the mixed protocol is more stable. CIC-IDS2017 provides a larger enterprise-network benchmark; Edge-IIoTset and ToN-IoT provide IoT/IIoT-oriented network traffic benchmarks. All datasets are reduced to binary normal/attack labels using documented preprocessing rules.
The evaluation is organized around four questions: primary performance under stable partitions, robustness under controlled label skew, the effect of server assistance and aggregation choices, and attack-type generalization. Table 2 reports the primary comparison. Table 3 reports alpha sensitivity for NSL-KDD and ToN-IoT. Table 4, Table 5 and Table 6 report schedule, aggregation, and teacher/budget controls. Table 7 reports Edge-IIoTset leave-one-attack testing, and Table 8 reports AUROC and AUPRC. All analyses use a common leakage-controlled validation protocol.
For Edge-IIoTset, high-cardinality identifiers, payload-like fields, and the multiclass Attack_type field are excluded before binary detection. For ToN-IoT, the Train_Test_Network.csv network-telemetry file is used. Label columns, timestamps, identifiers, IP addresses, ports, and high-cardinality application metadata are removed before model fitting. The final 80/20 stratified train–test split is made before the 10% server-validation and 10% client-validation splits, thereby preventing direct identifier memorization and preventing test labels from entering training, selection, or aggregation. The five evaluation settings differ substantially in their normal-to-anomalous sample composition, which provides a practical source of distributional variation for the subsequent evaluation (Figure 3).
The implementation uses Ubuntu 22.04, Python 3.10, and PyTorch 2.5.1 on a single GPU with 64 GB memory. The main experiments use K = 5 clients, at most R = 10 communication rounds, and a batch size of 2048. All methods use the same client partitions, seeds, detector backbone, maximum rounds, and final-test split within each protocol. Primary comparisons and alpha-sensitivity sweeps use three seeds (42, 2025, and 3407). Selected severe-heterogeneity controls use five seeds (42, 777, 1234, 2025, and 3407), specifically the schedule, teacher/budget, and ToN-IoT aggregation controls reported in Table 4, Table 5, Table 6, Table A3, Table A4 and Table A5. Results are reported as mean (standard deviation); Table A5 also reports t-based two-sided 95% confidence intervals for representative five-seed strong-skew controls. For the five-seed NSL-KDD strong-skew comparison, paired Wilcoxon tests with Holm correction did not yield adjusted p < 0.05 across the tested method pairs; we therefore avoid claims of statistical superiority based on small numerical differences.
Every run first fixes the final train–test split. A stratified server-validation subset is then separated from the training data, and each client retains a stratified local validation subset. Server validation controls early stopping and checkpoint selection; local validation F1 controls quality-aware aggregation; neither mechanism accesses the final-test set. Baselines receive the same backbone, partitions, maximum rounds, and final-test protocol. The proposed method additionally has access to the server-assisted teacher pool, so the no-teacher and budget-matched controls are reported alongside FLKD and the FD-IDS-style surrogate.
The comparison set spans standard averaging (FedAvg), proximal regularization (FedProx), local batch-normalization personalization (FedBN), adaptive server optimization (FedAdam), drift correction (SCAFFOLD), model-contrastive regularization (MOON), prototype transfer (FedProto), single-stage federated distillation (FLKD), and an FD-IDS-style surrogate. The FD-IDS result is a compatible surrogate implementation under the common backbone and protocol, not a reproduction of the original authors’ code. This label is retained throughout the manuscript and tables.

4.2. Main Performance Comparison

Table 2 shows that performance is setting-dependent. The proposed method has the highest mean F1 on CIC-IDS2017, NSL-KDD mixed, and ToN-IoT, while FedAdam is higher on NSL-KDD official and FedProto is higher on balanced Edge-IIoTset. On ToN-IoT, the difference between the proposed method and conventional baselines is small under the balanced partition, so it should be interpreted as competitive rather than as a universal advantage. This pattern motivates the strong-skew and control analyses below. This pattern is more readily interpreted by comparing the F1-score profiles across all evaluation settings in Figure 4.

4.3. Non-IID Robustness Analysis

As alpha decreases, client label distributions become more heterogeneous. On NSL-KDD official, the proposed method is competitive at alpha = 0.5 and alpha = 0.3, and it maintains a materially higher mean F1 than FedAvg and FedProx at alpha = 0.1. For ToN-IoT at alpha = 0.1, the five-seed comparison gives 91.46 ± 5.54 F1 for Ours, compared with 53.73 ± 49.00 for FedAvg and 53.77 ± 48.92 for FedProx. These results support a robustness-oriented interpretation, not a claim that all competitor families fail in every non-IID condition.
ToN-IoT broadens the empirical scope beyond the older NSL-KDD and CIC-IDS2017 corpora. It is still a benchmark with a stratified split and synthetic client partitioning in this study, so it does not substitute for device-, service-, protocol-, or time-defined deployment domains. Edge-IIoTset leave-one-attack testing remains the closest attack-shift analysis in this study; future work should add real device or temporal client partitions and cross-site transfer.
Figure 5 visualizes the alpha sweeps for the proposed method, FedAvg, and FedProx. The figure highlights the sharp variance growth of sample-size aggregation under severe skew while preserving the uncertainty bars. It should be read together with Table 3 rather than as a significance claim.

4.4. Ablation Study

The schedule controls do not identify a universally optimal order. On NSL-KDD official, the joint objective has the highest mean F1, whereas on ToN-IoT the 2→1→3 schedule has the highest mean F1. The default 1→2→3 schedule remains a transparent pre-specified protocol used in the primary comparison, but the manuscript no longer claims that it is intrinsically optimal. The more defensible finding is that timing and schedule are material design variables whose effects depend on dataset and heterogeneity.
Figure 6 compares full, no-teacher, and budget-matched variants under severe skew. It makes the contribution of server assistance and additional local optimization visible without relying on final-test performance for either aggregation or selection.

4.4.1. Schedule and Aggregation Controls

Table 5 isolates aggregation behavior under strong ToN-IoT label skew. Quality-aware weighting attains 91.46 ± 5.54 F1, compared with 53.68 ± 49.31 for sample-size weighting and approximately 88.4 for the two mixed weighting rules. This is a within-framework ablation, not a proof that quality weighting dominates every federated optimizer. It nevertheless supports the use of held-out validation quality as a practical safeguard against unstable local updates in this protocol.

4.4.2. Teacher and Budget Controls

Table 6 separates the server-assisted teacher assumption from the staged local budget. On NSL-KDD official at alpha = 0.1, the full method obtains 76.57 ± 3.23 F1, the budget-matched variant obtains 75.43 ± 4.85, and the no-teacher variant obtains 55.20 ± 29.80. On ToN-IoT at alpha = 0.1, the corresponding values are 91.46 ± 5.54, 88.10 ± 8.60, and 83.68 ± 16.89. These controls indicate that both the server-assisted teacher and the training protocol matter in these settings, while they also confirm that the full method relies on an assumption unavailable in fully decentralized FL.

4.4.3. Edge-IIoTset Leave-One-Attack Evaluation

Table 7 provides a stricter attack-shift test by holding out complete attack types. The results are attack-dependent. The proposed method has a higher mean F1 on DDoS_HTTP and Password than the listed standard and distillation baselines, but the budget-matched control is slightly higher on SQL_injection. The large uncertainty on SQL_injection prevents a strong ordering claim. These outcomes strengthen the paper’s bounded interpretation: robustness is scenario-specific and should be evaluated under explicit shift protocols.

4.5. Ranking Metrics and Convergence

Table 8 reports threshold-independent ranking metrics for the proposed method on each primary dataset. AUROC and AUPRC remain high on CIC-IDS2017, Edge-IIoTset, and ToN-IoT, while the lower NSL-KDD official values are consistent with its train–test shift. Ranking metrics are presented as complementary evidence and are not used for model selection or aggregation. Figure 7 provides a graphical summary of the ranking metrics and the selected strong-heterogeneity comparison. Figure 8 visualizes the schedule-control results and reinforces that the effect of stage ordering is dataset-dependent.

4.6. Efficiency Analysis

The staged framework adds training-side computation for teacher fitting, distillation losses, and validation-quality scoring, but it does not change the deployed student architecture. Table 9 is retained as an implementation profile rather than an efficacy result. Its absolute timing values depend on hardware, data loading, and early stopping, so the main practical conclusion is limited to a train-time versus inference-time trade-off.

5. Discussion

The results support a bounded robustness-oriented interpretation. The proposed method is competitive on stable settings and shows more favorable mean F1 under several severe label-skew settings, especially on ToN-IoT. It is not consistently the best method: FedAdam is higher on NSL-KDD official, FedProto is higher on balanced Edge-IIoTset, and alternative schedules can exceed the default schedule in the control studies. The evidence therefore supports a robustness-oriented design option, not universal superiority or a fixed universal curriculum. The rationale for the ordering is therefore empirical and mechanism-oriented: the schedule controls test whether separating early local boundary stabilization, subsequent decision transfer, and later representation alignment is useful under the evaluated heterogeneous conditions. These experiments do not constitute a general theoretical proof of convergence, optimality, or superiority for the proposed ordering.
The teacher controls clarify both a mechanism and a limitation. The server-assisted teacher improves or stabilizes performance in the reported strong-skew settings, while no-teacher and budget-matched variants show that local optimization alone does not fully explain all observed differences. At the same time, fitting the teacher on a server-accessible union of client-training subsets is incompatible with a strictly decentralized deployment that never exposes client records to a server. A practical decentralized extension would require a public reference set, synthetic data, secure aggregation of teacher statistics, or another privacy-preserving teacher-construction mechanism. We do not claim to solve that deployment problem here. This server-assisted requirement is a scope boundary of the present framework rather than a general solution for fully decentralized federated learning.
From a systems perspective, the main cost is concentrated during training. The teacher model, distillation losses, and quality-score computation increase per-round computation, but they do not increase the deployed inference model size. After training, each edge gateway can use the same anomaly detector architecture as the baseline student model, so deployment latency and memory footprint remain suitable for edge-gateway traffic monitoring.
This study concerns network and edge-gateway intrusion detection, not physical-layer sensing. The evaluation spans NSL-KDD, CIC-IDS2017, Edge-IIoTset, and ToN-IoT, together with five-seed strong-skew controls, held-out server and client validation, no-teacher and budget-matched variants, aggregation and schedule controls, and attack-type holdouts. Limitations remain. The FD-IDS comparison is a compatible surrogate rather than an exact reproduction; primary and attack-holdout results use three seeds; no observed mean difference is presented as statistically significant unless supported by the reported test; and client heterogeneity is still largely simulated by label skew. Future work should use device-, protocol-, time-, and site-defined client domains and an alternative teacher source for fully decentralized FL.

6. Conclusions

This paper evaluates a server-assisted, three-stage federated distillation protocol for intrusion detection in heterogeneous IoT/edge networks. This study makes the teacher data-access assumption, held-out validation protocol, and quality-score source explicit. It also evaluates progressive scheduling, teacher removal, budget matching, quality-aware aggregation, attack-type shift, and a fourth public dataset, ToN-IoT, under a common leakage-controlled protocol.
The evidence is deliberately bounded. The proposed method is competitive in stable partitions and shows favorable robustness in several severe label-skew and attack-shift settings, including ToN-IoT at Dirichlet alpha = 0.1. However, competitor methods and alternative schedules outperform it in selected conditions. The results therefore support server-assisted staged distillation as a useful robustness-oriented option, while not establishing universal superiority, a universally optimal stage order, or applicability to fully decentralized deployments without an alternative teacher source. The results likewise do not provide a formal convergence or optimality guarantee for the staged schedule.

Author Contributions

Conceptualization, X.Y. and Z.L.; methodology, Z.L. and X.Y.; software, Z.L., Z.Z., Z.J. and X.L.; validation, Z.L., Z.Z., Z.J. and X.L.; formal analysis, Z.L.; investigation, Z.L., Z.Z., Z.J. and X.L.; resources, X.Y., Q.L. and Y.D.; data curation, Z.L. and X.L.; writing—original draft preparation, Z.L.; writing—review and editing, X.Y., Q.L. and Y.D.; visualization, Z.L. and Z.Z.; supervision, X.Y. and Y.D.; project administration, X.Y.; funding acquisition, X.Y. All authors have read and agreed to the published version of the manuscript.

Funding

This research was supported by the Youth Innovation Team Project of the Shaanxi Provincial Department of Education (Grant No. 2030100790); the Shaanxi Provincial Social Science Foundation Project (Grant No. 2024J280); the Open Project of the Shanxi Key Laboratory of Data Factor Innovation and Economic Decision Analysis (Grant No. 2025-KF-02); the 2024 State Grid Henan Cooperation Project (Grant No. 5217L0240004); and Shaanxi Natural Science Basic Research Program (Grant No. 2026JC-YBQN-0756).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

NSL-KDD, CIC-IDS2017, Edge-IIoTset, and ToN-IoT are publicly available from their original repositories. The experiment archive includes preprocessing scripts, binary-label conversion rules, high-cardinality field removal rules, train/test and validation split indices, client partitions, random seeds, hyperparameters, and per-run logs. The teacher pool is derived from the client-training subsets remaining after held-out server and local validation partitions are created. The FD-IDS comparison is provided as a compatible surrogate implementation under the common experimental backbone.

Acknowledgments

The authors acknowledge the use of language-editing and formatting assistance during manuscript preparation. The authors reviewed and edited all content and take full responsibility for the final manuscript.

Conflicts of Interest

Author Qiuyan Li was employed by the company Economic and Technological Research Institute of State Grid Henan Electric Power Company. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Abbreviations

FLFederated learning
IDSIntrusion detection system
non-IIDNon-independent and identically distributed
KDKnowledge distillation
FBRForward bias regularization
IFBRInverse forward bias regularization
MKDMulti-module knowledge distillation
FLDFeature-level distillation
AUPRCArea under the precision–recall curve
AUROCArea under the receiver operating characteristic curve
FLKDFederated learning with knowledge distillation
FD-IDSFederated distillation intrusion detection system
FedProtoFederated prototype learning

Appendix A. Hyperparameters

Table A1. Key hyperparameters and training settings for NSL-KDD and CIC-IDS2017.
Table A1. Key hyperparameters and training settings for NSL-KDD and CIC-IDS2017.
ParameterMeaningNSL-KDDCIC-IDS2017Note
KNumber of clients55Main comparison setting
RMaximum communication rounds1010Unified across methods
ELocal epochs33Unified across methods
BBase batch size20482048Resource-aware
τDistillation temperature4.03.0IDS_TEMPERATURE
alpha2Stage 2 distillation weight0.550.30IDS_ALPHA_STAGE2
alpha3Stage 3 distillation weight0.450.20IDS_ALPHA_STAGE3
λ1FBR weight0.100.10IDS_FBR_WEIGHT
λ2IFBR weight0.150.10IDS_IFBR_WEIGHT
λ3Effective KD weightDynamicDynamicTeacher-confidence adjusted
λ4, λ5MKD/FLD weight0.200.10CMD_WEIGHT
λ6Calibration weight0.020.01IDS_CALIBRATION_WEIGHT
λ7/μProximal coefficient5 × 10−45 × 10−4Enabled mainly in Stage 3
βTeacher mixing coefficient0.00.0EMA experiment uses 0.2
q_kClient quality scoreLocal validation F1Local validation F1Quality-aware aggregation
alpha_dDirichlet alpha1.0/0.5/0.3/0.1-Heterogeneity setting
Table A2. Edge-IIoTset-specific training settings.
Table A2. Edge-IIoTset-specific training settings.
ParameterSettingNote
DatasetEdge-IIoTsetIoT/IIoT-oriented benchmark; balanced for main comparison
Samples2,219,201 total1,615,643 normal; 603,558 anomalous
Split80/20 stratified1,775,360 train; 443,841 test
ClientsK = 5Edge gateways/sensing domains
Rounds/epochsR = 10, E = 3Same budget as main experiments
Batch size2048Resource-aware adjustment enabled
Strong heterogeneityDirichlet alpha = 0.1Used for Edge-IIoTset strong-heterogeneity ablation
Minimum client samples1Matched full and ablation settings
MetricsAccuracy, Precision, Recall, F1, AUROC, AUPRCReported over three seeds
Leave-one-attackPassword/DDoS_HTTP/SQL_injectionAttack_type used only for split construction and removed from model inputs
DatasetToN-IoTNetwork telemetry; 80/20 stratified split; 10% server and 10% local validation held out before client training

Appendix B. Supplementary Mechanism Experiments

Appendix B.1. Stage-Order Validation

Table A3. Alternative schedule controls under strong heterogeneity on NSL-KDD official and ToN-IoT. Values are mean (standard deviation) over five seeds.
Table A3. Alternative schedule controls under strong heterogeneity on NSL-KDD official and ToN-IoT. Values are mean (standard deviation) over five seeds.
Dataset Schedule F1 (%) n Validation Protocol Teacher Assumption Interpretation
NSL-KDD official1→2→3 (default)76.57 (3.23)5Held-out server/local validationServer-assisted training poolSchedule-dependent
1→3→276.36 (3.20)5Held-out server/local validationServer-assisted training poolSchedule-dependent
2→1→376.73 (4.75)5Held-out server/local validationServer-assisted training poolSchedule-dependent
Joint objective79.74 (5.01)5Held-out server/local validationServer-assisted training poolSchedule-dependent
ToN-IoT1→2→3 (default)91.46 (5.54)5Held-out server/local validationServer-assisted training poolSchedule-dependent
1→3→288.38 (8.82)5Held-out server/local validationServer-assisted training poolSchedule-dependent
2→1→393.02 (3.97)5Held-out server/local validationServer-assisted training poolSchedule-dependent
Joint objective90.55 (4.76)5Held-out server/local validationServer-assisted training poolSchedule-dependent
The schedule controls are dataset-dependent. Neither the default order nor the joint objective is universally best; the default is retained as a transparent protocol choice for the main comparison, and the alternative results are reported to avoid attributing all gains to one presumed ordering.

Appendix B.2. Teacher Update Strategy

Table A4. Server-assistance boundary and evaluation controls used in the protocol.
Table A4. Server-assistance boundary and evaluation controls used in the protocol.
Control Teacher Source Training Data
Exposure
Use in Selection/Aggregation Purpose Limit Status
Full methodServer-accessible client-training poolYes, simulation assumptionTeacher guidance; local validation for aggregationPrimary server-assisted protocolNot fully decentralizedReported
No-teacherNoneNo, teacher poolLocal validation for aggregationIsolate teacher contributionDoes not test alternative teacher sourcesReported
Budget-matchedServer-accessible client-training poolYes, simulation assumptionLocal validation for aggregationSeparate local optimization budgetStill server-assistedReported

Appendix B.3. Quality-Aware Aggregation

Figure A1 summarizes the ToN-IoT aggregation controls under strong heterogeneity. Together with Table A3 and Table A4, it documents schedule dependence, the server-assistance boundary, and the behavior of alternative aggregation rules.

Appendix B.4. Confidence Intervals and Paired Comparisons

Table A5. Two-sided t-based 95% confidence intervals for representative five-seed strong-heterogeneity F1 controls.
Table A5. Two-sided t-based 95% confidence intervals for representative five-seed strong-heterogeneity F1 controls.
Dataset and ProtocolMethod/ControlF1, Mean (SD)95% CI
NSL-KDD official, alpha = 0.1Ours76.57 (3.23)72.56–80.58
ToN-IoT, alpha = 0.1Ours, order 1→2→391.46 (5.54)84.58–98.34
ToN-IoT, alpha = 0.1Order 1→3→288.38 (8.82)77.43–99.33
ToN-IoT, alpha = 0.1Order 2→1→393.02 (3.97)88.09–97.95
ToN-IoT, alpha = 0.1Joint objective90.55 (4.76)84.64–96.46
Intervals use the Student t distribution with four degrees of freedom. Because the paired Wilcoxon tests with Holm correction on the five-seed NSL-KDD strong-skew comparison did not show adjusted p < 0.05, the manuscript treats these results as robustness evidence and does not interpret small mean differences as statistically confirmed superiority.
Figure A1. ToN-IoT aggregation-rule controls under Dirichlet alpha = 0.1.
Figure A1. ToN-IoT aggregation-rule controls under Dirichlet alpha = 0.1.
Electronics 15 03810 g0a1
For reproducibility, the archived experiment package records preprocessing, train/test and validation split indices, client partitions, server-assisted teacher-pool construction, random seeds, full hyperparameters, and run logs for all controls and baselines. These artifacts are necessary to distinguish final-test evaluation from training, validation, checkpoint selection, and quality scoring.

References

  1. Ferrag, M.A.; Friha, O.; Maglaras, L.; Janicke, H.; Shu, L. Federated deep learning for cyber security in the Internet of Things: Concepts, applications, and experimental analysis. IEEE Access 2021, 9, 138509–138542. [Google Scholar] [CrossRef] [Scilit]
  2. Buyuktanir, B.; Altinkaya, S.; Karatas Baydogmus, G.; Yildiz, K. Federated learning in intrusion detection: Advancements, applications, and future directions. Clust. Comput. 2025, 28, 473. [Google Scholar] [CrossRef] [Scilit]
  3. Makris, I.; Karampasi, A.; Radoglou-Grammatikis, P.; Episkopos, N.; Iturbe, E.; Rios, E.; Piperigkos, N.; Lalos, A.; Xenakis, C.; Lagkas, T.; et al. A comprehensive survey of federated intrusion detection systems: Techniques, challenges and solutions. Comput. Sci. Rev. 2025, 56, 100717. [Google Scholar] [CrossRef] [Scilit]
  4. Ferrag, M.A.; Friha, O.; Hamouda, D.; Maglaras, L.; Janicke, H. Edge-IIoTset: A New Comprehensive Realistic Cyber Security Dataset of IoT and IIoT Applications for Centralized and Federated Learning. IEEE Access 2022, 10, 40281–40306. [Google Scholar] [CrossRef] [Scilit]
  5. Koroniotis, N.; Moustafa, N.; Sitnikova, E.; Turnbull, B. Towards the development of realistic botnet dataset in the Internet of Things for network forensic analytics: Bot-IoT dataset. Future Gener. Comput. Syst. 2019, 100, 779–796. [Google Scholar] [CrossRef] [Scilit]
  6. Alsaedi, A.; Moustafa, N.; Tari, Z.; Mahmood, A.; Anwar, A. ToN-IoT Telemetry Dataset: A New Generation Dataset of IoT and IIoT for Data-Driven Intrusion Detection Systems. IEEE Access 2020, 8, 165130–165150. [Google Scholar] [CrossRef] [Scilit]
  7. Meidan, Y.; Bohadana, M.; Mathov, Y.; Mirsky, Y.; Shabtai, A.; Breitenbacher, D.; Elovici, Y. N-BaIoT-Network-based Detection of IoT Botnet Attacks Using Deep Autoencoders. IEEE Pervasive Comput. 2018, 17, 12–22. [Google Scholar] [CrossRef] [Scilit]
  8. Li, T.; Sahu, A.K.; Zaheer, M.; Sanjabi, M.; Talwalkar, A.; Smith, V. Federated optimization in heterogeneous networks. Proc. Mach. Learn. Syst. 2020, 2, 429–450. [Google Scholar]
  9. Li, X.; Jiang, M.; Zhang, X.; Kamp, M.; Dou, Q. FedBN: Federated learning on non-IID features via local batch normalization. In Proceedings of the International Conference on Learning Representations, Virtual, 3–7 May 2021. [Google Scholar]
  10. McMahan, H.B.; Moore, E.; Ramage, D.; Hampson, S.; Arcas, B.A. Communication-efficient learning of deep networks from decentralized data. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, Fort Lauderdale, FL, USA, 20–22 April 2017; PMLR: New York, NY, USA, 2017; pp. 1273–1282. [Google Scholar]
  11. Karimireddy, S.P.; Kale, S.; Mohri, M.; Reddi, S.J.; Stich, S.U.; Suresh, A.T. SCAFFOLD: Stochastic Controlled Averaging for Federated Learning. In Proceedings of the 37th International Conference on Machine Learning, Virtual, 13–18 July 2020; PMLR: New York, NY, USA, 2020; pp. 5132–5143. [Google Scholar]
  12. Reddi, S.J.; Charles, Z.; Zaheer, M.; Garrett, Z.; Rush, K.; Konecny, J.; Kumar, S.; McMahan, H.B. Adaptive Federated Optimization. In Proceedings of the 9th International Conference on Learning Representations, Virtual, 3–7 May 2021. [Google Scholar]
  13. Hinton, G.; Vinyals, O.; Dean, J. Distilling the knowledge in a neural network. arXiv 2015, arXiv:1503.02531. [Google Scholar]
  14. Gou, J.; Yu, B.; Maybank, S.J.; Tao, D. Knowledge distillation: A survey. Int. J. Comput. Vis. 2021, 129, 1789–1819. [Google Scholar] [CrossRef] [Scilit]
  15. Tian, Y.; Krishnan, D.; Isola, P. Contrastive representation distillation. In Proceedings of the International Conference on Learning Representations, Addis Ababa, Ethiopia, 26–30 April 2020. [Google Scholar]
  16. Peng, H.; Wu, C.; Xiao, Y. FD-IDS: Federated Learning with Knowledge Distillation for Intrusion Detection in Non-IID IoT Environments. Sensors 2025, 25, 4309. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  17. Li, Q.; He, B.; Song, D. Model-Contrastive Federated Learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 20–25 June 2021; IEEE: New York, NY, USA, 2021; pp. 10713–10722. [Google Scholar] [CrossRef] [Scilit]
  18. Lin, T.; Kong, L.; Stich, S.U.; Jaggi, M. Ensemble Distillation for Robust Model Fusion in Federated Learning. In Proceedings of the Advances in Neural Information Processing Systems 33, Virtual, 6–12 December 2020. [Google Scholar]
  19. Tan, Y.; Long, G.; Liu, L.; Zhou, T.; Lu, Q.; Jiang, J.; Zhang, C. FedProto: Federated Prototype Learning across Heterogeneous Clients. Proc. AAAI Conf. Artif. Intell. 2022, 36, 8432–8440. [Google Scholar] [CrossRef] [Scilit]
  20. Wang, J.; Yang, T.; Chen, W.; Deng, H.; Huang, Q.; Li, D. Federated learning with self-updating server parameters for smart home intrusion detection in non-IID environments. Expert Syst. Appl. 2025, 267, 126233. [Google Scholar] [CrossRef] [Scilit]
  21. Zheng, X.; Wu, W.; Yang, S.; Wang, X. Robust federated intrusion detection under statistical heterogeneity. Comput. Netw. 2026, 275, 111904. [Google Scholar] [CrossRef] [Scilit]
  22. Tavallaee, M.; Bagheri, E.; Lu, W.; Ghorbani, A.A. A detailed analysis of the KDD CUP 99 data set. In Proceedings of the 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications, Ottawa, ON, Canada, 8–10 July 2009; IEEE: New York, NY, USA, 2009; pp. 1–6. [Google Scholar] [CrossRef] [Scilit]
  23. Sharafaldin, I.; Habibi Lashkari, A.; Ghorbani, A.A. Toward generating a new intrusion detection dataset and intrusion traffic characterization. In Proceedings of the 4th International Conference on Information Systems Security and Privacy, Funchal, Portugal, 22–24 January 2018; SciTePress: Setúbal, Portugal, 2018; pp. 108–116. [Google Scholar] [CrossRef] [Scilit]
  24. Mothukuri, V.; Parizi, R.M.; Pouriyeh, S.; Huang, Y.; Dehghantanha, A.; Srivastava, G. Federated-learning-based anomaly detection for IoT security attacks. IEEE Internet Things J. 2022, 9, 2545–2554. [Google Scholar] [CrossRef] [Scilit]
  25. Chen, Z.; Lv, N.; Liu, P.; Fang, Y.; Chen, K.; Pan, W. Intrusion detection for wireless edge networks based on federated learning. IEEE Access 2020, 8, 217463–217472. [Google Scholar] [CrossRef] [Scilit]
  26. Li, B.; Wu, Y.; Song, J.; Lu, R.; Li, T.; Zhao, L. DeepFed: Federated deep learning for intrusion detection in industrial cyber-physical systems. IEEE Trans. Ind. Inform. 2021, 17, 5615–5624. [Google Scholar] [CrossRef] [Scilit]
Figure 1. Overall framework of the proposed three-stage federated distillation method.
Figure 1. Overall framework of the proposed three-stage federated distillation method.
Electronics 15 03810 g001
Figure 2. Training sequence within one communication round. Blue, green, and orange regions denote the server, client, and teacher-reference components, respectively; solid arrows indicate model or parameter transmission, whereas dashed arrows indicate teacher-provided signals.
Figure 2. Training sequence within one communication round. Blue, green, and orange regions denote the server, client, and teacher-reference components, respectively; solid arrows indicate model or parameter transmission, whereas dashed arrows indicate teacher-provided signals.
Electronics 15 03810 g002
Figure 3. Normal and anomalous training sample distributions across the five evaluation settings. The ToN-IoT counts correspond to its stratified training partition.
Figure 3. Normal and anomalous training sample distributions across the five evaluation settings. The ToN-IoT counts correspond to its stratified training partition.
Electronics 15 03810 g003
Figure 4. F1-score comparison under the leakage-controlled primary protocol across the five evaluation settings.
Figure 4. F1-score comparison under the leakage-controlled primary protocol across the five evaluation settings.
Electronics 15 03810 g004
Figure 5. F1-score sensitivity to Dirichlet heterogeneity on NSL-KDD official and ToN-IoT under the leakage-controlled protocol.
Figure 5. F1-score sensitivity to Dirichlet heterogeneity on NSL-KDD official and ToN-IoT under the leakage-controlled protocol.
Electronics 15 03810 g005
Figure 6. Server-assistance and local-budget controls under strong heterogeneity on NSL-KDD official and ToN-IoT.
Figure 6. Server-assistance and local-budget controls under strong heterogeneity on NSL-KDD official and ToN-IoT.
Electronics 15 03810 g006
Figure 7. Ranking-quality and strong-heterogeneity summaries under the leakage-controlled protocol: (a) AUROC and (b) AUPRC on the primary settings; (c) selected baselines on ToN-IoT with Dirichlet alpha = 0.1.
Figure 7. Ranking-quality and strong-heterogeneity summaries under the leakage-controlled protocol: (a) AUROC and (b) AUPRC on the primary settings; (c) selected baselines on ToN-IoT with Dirichlet alpha = 0.1.
Electronics 15 03810 g007
Figure 8. Schedule controls under strong heterogeneity on NSL-KDD official and ToN-IoT. The panels demonstrate dataset-dependent schedule behavior rather than a universal optimal ordering.
Figure 8. Schedule controls under strong heterogeneity on NSL-KDD official and ToN-IoT. The panels demonstrate dataset-dependent schedule behavior rather than a universal optimal ordering.
Electronics 15 03810 g008
Table 1. Canonical terminology and roles of the proposed modules.
Table 1. Canonical terminology and roles of the proposed modules.
ComponentCanonical TermRoleEvidence Reported
Stage 1Boundary stabilizationLimits early local prediction bias using a balanced-output regularizer.Schedule controls; severe label-skew tests.
Stage 2Decision distillationTransfers confidence-weighted teacher soft targets after initial local fitting.No-teacher and budget-matched controls.
Stage 3Representation alignmentAligns teacher and student intermediate features and their first-block statistics.Alternative schedule controls.
ServerValidation-quality-aware aggregationWeights update by held-out local validation F1, not final-test performance.Aggregation-rule ablation on ToN-IoT.
TeacherServer-assisted referenceFits a teacher on a server-accessible pool of client-training subsets in the simulation.Explicit limitation; not a fully decentralized deployment claim.
Table 2. Main method comparison under the leakage-controlled primary protocol. Values are mean (standard deviation) over three seeds.
Table 2. Main method comparison under the leakage-controlled primary protocol. Values are mean (standard deviation) over three seeds.
DatasetMethodAccuracy (%)F1 (%)AUROCAUPRC
NSL-KDD officialFedAdam80.2680.52 (0.73)0.8160.820
Ours78.9478.72 (0.21)0.8600.917
FedProto79.0078.63 (0.50)0.8650.913
FedProx78.9178.59 (0.51)0.8620.912
FedAvg78.9278.59 (0.49)0.8620.912
FLKD78.8078.51 (0.32)0.8590.916
MOON78.6878.45 (0.48)0.8940.914
FedBN78.3077.96 (0.45)0.8690.923
SCAFFOLD74.6973.31 (1.45)0.9300.943
NSL-KDD mixedOurs98.7198.66 (0.06)0.9990.999
FedProto98.6898.63 (0.07)0.9990.999
FedAvg98.6598.60 (0.03)0.9990.999
FedProx98.6598.60 (0.03)0.9990.999
FLKD98.5898.53 (0.06)0.9990.999
MOON98.4198.35 (0.14)0.9990.998
FedBN97.0196.88 (0.32)0.9970.997
SCAFFOLD96.2896.09 (0.10)0.9920.991
FedAdam92.0291.83 (1.77)0.9210.879
CIC-IDS2017Ours97.9895.08 (0.60)0.9980.994
FLKD97.8894.82 (0.11)0.9980.992
MOON97.6494.26 (0.59)0.9980.991
FedAvg97.3193.57 (0.48)0.9980.992
FedProx97.3093.54 (0.30)0.9980.992
FedProto97.2693.45 (0.09)0.9980.992
SCAFFOLD96.6191.95 (0.05)0.9960.985
FedBN96.7391.94 (0.93)0.9940.981
FedAdam69.1252.70 (18.87)0.7200.381
Edge-IIoTsetFedProto98.6597.49 (0.82)0.9990.998
Ours98.2396.67 (2.12)0.9970.994
MOON97.5595.35 (2.11)0.9960.992
FLKD97.5495.35 (0.16)0.9980.995
FedAvg97.4995.28 (0.95)0.9960.992
FedProx97.4495.20 (1.12)0.9950.991
SCAFFOLD96.0492.43 (0.28)0.9790.967
FedBN87.0181.55 (15.04)0.9600.940
FedAdam64.3657.58 (9.02)0.7010.423
ToN-IoTOurs91.4394.10 (0.05)0.9880.996
FedAvg91.3894.06 (0.09)0.9900.997
FedProx91.3694.04 (0.07)0.9900.997
FedProto91.3394.02 (0.10)0.9890.997
FLKD91.3194.01 (0.08)0.9860.996
FD-IDS surrogate91.2893.99 (0.05)0.9850.995
MOON91.2893.99 (0.11)0.9820.994
SCAFFOLD91.2193.94 (0.09)0.9800.994
FedAdam81.5988.19 (0.81)0.7220.858
FedBN82.3685.91 (13.95)0.9860.995
Table 3. F1-score sensitivity to Dirichlet heterogeneity on NSL-KDD official and ToN-IoT. Values are mean (standard deviation); n indicates the number of seeds.
Table 3. F1-score sensitivity to Dirichlet heterogeneity on NSL-KDD official and ToN-IoT. Values are mean (standard deviation); n indicates the number of seeds.
DatasetDirichlet AlphaMethodF1 (%)n
NSL-KDD official1.0Ours78.68 (0.14)3
FedAvg79.41 (0.48)3
FedProx79.32 (0.58)3
0.5Ours79.29 (0.55)3
FedAvg77.28 (1.71)3
FedProx77.54 (1.51)3
0.3Ours77.94 (2.54)3
FedAvg77.01 (14.16)3
FedProx77.45 (13.71)3
0.1Ours77.14 (3.54)3
FedAvg50.93 (42.51)3
FedProx50.82 (42.91)3
ToN-IoT1.0Ours94.04 (0.12)3
FedAvg94.04 (0.07)3
FedProx94.04 (0.05)3
0.5Ours94.11 (0.23)3
FedAvg94.04 (0.12)3
FedProx94.04 (0.12)3
0.3Ours94.11 (0.24)3
FedAvg85.61 (13.67)3
FedProx81.39 (21.96)3
0.1Ours91.46 (5.54)5
FedAvg53.73 (49.00)5
FedProx53.77 (48.92)5
Table 4. Schedule controls under strong heterogeneity (Dirichlet alpha = 0.1). Values are mean (standard deviation) over five seeds.
Table 4. Schedule controls under strong heterogeneity (Dirichlet alpha = 0.1). Values are mean (standard deviation) over five seeds.
DatasetScheduleF1 (%)n
NSL-KDD official1→2→3 (default)76.57 (3.23)5
1→3→276.36 (3.20)5
2→1→376.73 (4.75)5
Joint objective79.74 (5.01)5
ToN-IoT1→2→3 (default)91.46 (5.54)5
1→3→288.38 (8.82)5
2→1→393.02 (3.97)5
Joint objective90.55 (4.76)5
Table 5. Aggregation-rule controls on ToN-IoT under Dirichlet alpha = 0.1. Values are mean (standard deviation) over five seeds.
Table 5. Aggregation-rule controls on ToN-IoT under Dirichlet alpha = 0.1. Values are mean (standard deviation) over five seeds.
AggregationAccuracy (%)F1 (%)AUROCAUPRCEvaluation Sourcen
Sample size60.9853.68 (49.31)0.9590.987Held-out local validation5
Sample x quality84.2588.42 (10.01)0.9200.964Held-out local validation5
Clipped quality83.9488.49 (9.81)0.9130.961Held-out local validation5
Quality-aware88.1791.46 (5.54)0.9670.988Held-out local validation5
Table 6. Teacher and budget controls under strong heterogeneity. Values are mean (standard deviation) over five seeds.
Table 6. Teacher and budget controls under strong heterogeneity. Values are mean (standard deviation) over five seeds.
DatasetMethod/ControlAccuracy (%)F1 (%)AUROCAUPRC
NSL-KDD officialOurs75.2076.57 (3.23)0.8820.926
Ours w/o teacher66.6255.20 (29.80)0.7870.869
Ours budget-matched76.7975.43 (4.85)0.8330.898
FLKD67.3761.65 (33.58)0.8230.884
ToN-IoTOurs88.1791.46 (5.54)0.9670.988
Ours w/o teacher79.2683.68 (16.89)0.8700.940
Ours budget-matched83.2788.10 (8.60)0.8760.944
FLKD58.8252.39 (47.76)0.9450.976
FD-IDS surrogate59.6752.96 (48.21)0.9570.978
Table 7. Edge-IIoTset leave-one-attack evaluation under the leakage-controlled protocol. The held-out attack type is used only for final testing; values are mean (standard deviation) over three seeds.
Table 7. Edge-IIoTset leave-one-attack evaluation under the leakage-controlled protocol. The held-out attack type is used only for final testing; values are mean (standard deviation) over three seeds.
Held-Out AttackMethodF1 (%)AUROCAUPRC
DDoS_HTTPOurs86.51 (9.02)0.9600.916
FD-IDS surrogate80.53 (17.27)0.9510.894
FedAvg80.26 (17.50)0.9510.886
FedProx79.43 (12.91)0.9480.875
Ours budget-matched79.32 (19.99)0.9520.886
Ours w/o teacher70.07 (24.33)0.9600.868
SCAFFOLD57.16 (32.48)0.9080.777
PasswordOurs72.20 (0.85)0.8800.788
Ours budget-matched71.47 (2.72)0.8640.763
Ours w/o teacher68.24 (8.20)0.9130.780
SCAFFOLD40.63 (29.34)0.7530.462
FedAvg39.49 (31.13)0.6770.446
FedProx39.34 (30.86)0.6730.440
FD-IDS surrogate32.53 (34.69)0.6680.431
SQL_injectionOurs budget-matched77.27 (31.95)0.9400.867
Ours76.52 (34.17)0.9040.839
Ours w/o teacher70.89 (23.57)0.8980.804
FD-IDS surrogate63.40 (27.85)0.9510.803
SCAFFOLD31.57 (12.98)0.8570.609
FedProx23.72 (7.95)0.9860.913
FedAvg21.93 (11.50)0.9870.921
Table 8. AUROC and AUPRC for the proposed method under the leakage-controlled primary protocol. Values are mean (standard deviation) over three seeds.
Table 8. AUROC and AUPRC for the proposed method under the leakage-controlled primary protocol. Values are mean (standard deviation) over three seeds.
DatasetMethodAUROCAUPRC
NSL-KDD officialOurs0.8600.917
NSL-KDD mixedOurs0.9990.999
CIC-IDS2017Ours0.9980.994
Edge-IIoTsetOurs0.9970.994
ToN-IoTOurs0.9880.996
Table 9. Training and inference cost on NSL-KDD official. This implementation-level profile is reported separately from the leakage-controlled performance comparisons.
Table 9. Training and inference cost on NSL-KDD official. This implementation-level profile is reported separately from the leakage-controlled performance comparisons.
MethodTotal Time (s)Time/Round (s)Latency (ms/Batch)Latency (ms/Sample)Throughput (Samples/s)Peak Memory (MB)Extra Inference Params
Ours892.79 (371.57)120.74 (31.72)5.1600.003363,9839.830
FedAvg933.86 (211.67)117.21 (2.89)6.1700.003304,5089.830
FedProx817.06 (179.51)117.38 (3.97)5.1600.003363,8489.830
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Yang, X.; Lin, Z.; Li, Q.; Dong, Y.; Zhang, Z.; Jing, Z.; Lu, X. A Three-Stage Federated Distillation Framework for Robust Intrusion Detection in Heterogeneous IoT/Edge Networks. Electronics 2026, 15, 3810. https://doi.org/10.3390/electronics15173810

AMA Style

Yang X, Lin Z, Li Q, Dong Y, Zhang Z, Jing Z, Lu X. A Three-Stage Federated Distillation Framework for Robust Intrusion Detection in Heterogeneous IoT/Edge Networks. Electronics. 2026; 15(17):3810. https://doi.org/10.3390/electronics15173810

Chicago/Turabian Style

Yang, Xudong, Ziyi Lin, Qiuyan Li, Yuanxiang Dong, Zhenyu Zhang, Zhenzhou Jing, and Xuyao Lu. 2026. "A Three-Stage Federated Distillation Framework for Robust Intrusion Detection in Heterogeneous IoT/Edge Networks" Electronics 15, no. 17: 3810. https://doi.org/10.3390/electronics15173810

APA Style

Yang, X., Lin, Z., Li, Q., Dong, Y., Zhang, Z., Jing, Z., & Lu, X. (2026). A Three-Stage Federated Distillation Framework for Robust Intrusion Detection in Heterogeneous IoT/Edge Networks. Electronics, 15(17), 3810. https://doi.org/10.3390/electronics15173810

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Article metric data becomes available approximately 24 hours after publication online.
Back to TopTop