Adapting the IDS-ML Framework for Automated Attack Detection on Edge Devices

Abstract

As modern networks expand, the volume and destructiveness of cyberattacks continue to escalate, necessitating effective defense mechanisms. Intrusion Detection Systems (IDSs) are critical for maintaining network security; however, traditional signature-based systems often fail to detect zero-day attacks. This study explores recent advancements in Deep Learning (DL) for cybersecurity by analyzing and replicating the “IDS-ML” framework, an open-source repository for IDS development. We evaluate the performance of five deep learning Convolutional Neural Network (CNN) architectures adapted for intrusion detection via transfer learning on the CICIDS2017 dataset, and propose an enhancement by integrating Automated Machine Learning (AutoML) techniques that achieves a 94.7% reduction in model parameters while maintaining comparable accuracy, thus making our enhanced models suitable for deployment on edge devices. We further validate deployment feasibility by benchmarking both the baseline InceptionV3 and AutoML models on a Raspberry Pi 4, demonstrating an 18.7× inference speedup and 3.5× CPU reduction, with no change in predicted classes from model conversion. Our results confirm that lightweight AutoML architectures enable practical “zero-touch” edge-based intrusion detection on resource-constrained hardware.

1. Introduction

Network security has become a central concern as internet and communication technologies have expanded. More applications running over modern networks mean an increased attack surface and a larger blast radius when intrusions succeed [1]. Internet of Things (IoT) and Operational Technology (OT) devices account for a growing share of successful intrusions [2], and the consequences in high-stakes OT settings extend beyond data loss: a compromised substation or distributed control device in smart grid infrastructure can disrupt essential services in the physical world [3]. Intrusion Detection Systems (IDSs) are the primary defensive layer against these threats, monitoring network traffic to flag abnormal or malicious activity.

Traditionally, IDSs are categorized into signature-based and anomaly-based systems. Signature-based systems are effective at detecting known attacks but struggle with “zero-day” attacks and new threats with unknown patterns [4]. They also depend on continuous signature database updates, which is impractical in many OT environments where frequent updates introduce operational risk and downtime. This motivates a “zero-touch” model of defense in which detection capabilities are embedded directly on the edge device and operate autonomously without relying on signature refreshes or cloud offload [5]. Anomaly-based systems can detect these unknown attacks but often suffer from higher false positive rates [6]. Machine Learning (ML) has been widely applied to bridge this gap, but developing effective ML-based IDSs involves complex challenges. Specifically, manual hyperparameter optimization (HPO) is time consuming and requires significant expert knowledge to define search spaces effectively [7].

A further gap in the ML-IDS literature is that most work reports classification metrics on benchmark datasets without quantifying deployment feasibility on resource-constrained hardware. Recent surveys emphasize that lightweight, efficient IDS models suitable for edge deployment remain an open research need [8]. Automated Machine Learning (AutoML) is particularly well suited to close this gap because it can jointly optimize for classification accuracy and model efficiency, producing compact architectures that manual tuning rarely yields [5].

This paper focuses on the analysis, replication, and enhancement of the IDS-ML framework proposed by Yang and Shami [1]. The IDS-ML repository provides a comprehensive suite of ML algorithms, including tree-based models and Convolutional Neural Networks (CNN) via transfer learning. The contributions of this work are:

1.

Reproduction: We reproduce five deep learning CNN architectures (VGG16, VGG19, Xception, InceptionV3, and InceptionResNetV2) adapted for intrusion detection via transfer learning, as proposed in the IDS-ML framework [1], to establish a performance baseline on the CICIDS2017 dataset. Whereas the original IDS-ML framework used Bayesian Optimization to tune the training hyperparameters of each baseline, our reproduction uses fixed defaults. This affects only training behavior, not the architectures’ parameter counts; Bayesian Optimization in the original setup tuned training behavior within fixed transfer-learning architectures and therefore would not have reduced the baselines’ model size, so the model compression finding that motivates this paper is unchanged. Comparative results across all five architectures are reported in Section 4, from which InceptionV3 emerged as the highest-performing baseline, a finding that matches the ranking reported in the original IDS-ML paper [1], and was therefore selected as the reference model for subsequent AutoML comparison and edge deployment validation.

2.

Enhancement: We propose an AutoML-driven pipeline to replace the manual HPO steps. Recent studies in 2024 and 2025 indicate that AutoML can improve detection efficiency in resource-constrained environments by automating feature selection [5].

3.

Evaluation: We provide a comparative analysis of the reproduced baselines against our AutoML-discovered architecture, focusing on the trade-off between classification performance and trainable parameter count. This trade-off determines whether a model can be deployed on gateway class edge hardware: even highly accurate baselines are operationally infeasible if their memory or inference cost exceeds the constraints of the target device.

4.

Edge Validation: We deploy both the InceptionV3 baseline and AutoML model on a Raspberry Pi 4 to empirically measure inference latency, CPU utilization, memory consumption, and thermal behavior under sustained network traffic loads, quantifying the operational advantage of model compression on resource-constrained hardware.

Together, these contributions yield an AutoML architecture that is 94.7% smaller than the InceptionV3 baseline (1.18 M vs. 22.3 M parameters) while preserving classification performance (Weighted F1 of 0.9203 vs. 0.9186), and delivers an 18.7× inference speedup with 3.5× less CPU on a Raspberry Pi 4 under sustained network traffic loads.

The rest of the paper is organized as follows. Section 2 reviews related work on ML-based IDS and edge deployment. Section 3 describes the dataset, experimental setup, and the three-phase methodology. Section 4 presents the replication, AutoML enhancement, and hardware benchmarking results. Section 5 discusses the significance of model compression, the AutoML advantage, and edge deployment implications. Section 6 concludes and outlines future work.

2. Related Work

2.1. Machine Learning and Deep Learning in IDS

Recent literature categorizes ML algorithms into supervised and unsupervised learning. Supervised algorithms, such as Decision Trees (DT) and K-Nearest Neighbors (KNN), are typically used for signature-based detection [1]. Deep Learning has further advanced this field, with CNNs showing promise in extracting complex features from high-dimensional network traffic data [9]. For instance, pre-trained image recognition architectures such as VGG16 [10] have been adapted to network traffic classification via transfer learning [1].

2.2. Zero-Day Attack Detection

The defining challenge for modern IDS is the detection of zero-day attacks, for which no prior signature or label exists. Signature-based systems are inherently blind to these threats [4], shifting the burden to anomaly-based and learning-based approaches that identify deviations from learned behavioral baselines rather than matching known patterns. A recent survey by Guo [11] categorizes contemporary approaches into unsupervised, semi-supervised, and transfer learning families, and identifies training data representativeness and detection consistency as the principal open challenges.

Among unsupervised approaches, autoencoder-based detectors have become a dominant paradigm. These models are trained exclusively on benign traffic and flag samples that produce high reconstruction error as anomalous, under the assumption that novel attack traffic lies outside the pattern of learned normal behavior. Despite these advances, purely anomaly-based systems remain susceptible to high false positive rates in diverse production traffic [6], thus motivating hybrid architectures that combine supervised classification of known attacks with anomaly-based coverage. Recent work in this direction includes attention-augmented hybrid models such as the LSTM-CNN-attention architecture of Gueriani et al. [12], which couples temporal and spatial feature learning with attention weighting and applies oversampling to improve detection on imbalanced IIoT traffic, or transformer-based detectors such as IDS-INT [13] which pairs transformer transfer learning with CNN-LSTM classification and SMOTE balancing for imbalanced network traffic. These models report high accuracy but like most of the deep learning IDS literature are not evaluated for inference cost on constrained edge hardware.

2.3. Edge-Based Intrusion Detection

Deploying deep learning IDS on resource-constrained edge devices requires techniques that reduce the computational, memory, and energy footprint of trained models. Liang et al. [14] surveyed the principal compression strategies, grouping them into weight pruning (eliminating redundant parameters), quantization (reducing numeric precision from FP32 to INT8 or below [15]), knowledge distillation (training a smaller student network to mimic a larger teacher), and low rank decomposition. Each technique trades some degree of accuracy for improved inference efficiency, and they are frequently combined in deployment pipelines [16].

Raspberry Pi class single-board computers have emerged as the reference platform for evaluating compressed CNN-based IDS, offering sufficient memory and compute headroom to run full TFLite inference under realistic traffic loads while retaining the form factor and power profile of production IoT gateways [17,18]. Recent IoT IDS surveys highlight that lightweight, efficient models suitable for such resource-constrained deployment remain an open research need [8], which this work directly addresses by benchmarking both compressed and baseline architectures on identical edge hardware.

Most recent high-accuracy IDS research pursues a different objective than edge deployability. Transformer- and attention-based detectors such as RTIDS [19], the modular FlowTransformer framework [20], the improved Vision Transformer of Yang et al. [21], and hybrid CNN-LSTM and CNN-LSTM-attention models [12,13,22] report near perfect accuracy and F1 on benchmark datasets, but are evaluated on workstation or server class hardware and generally do not report parameter counts. This line of work is best viewed as a parallel and complementary direction rather than a competitor to the present study since it advances detection performance whereas this work targets fitting a usable model within the resource constraints of an edge device.

Table 1. Positioning of this work relative to recent deep learning IDS research. “Reported Acc.” is each cited paper’s own best result on the dataset it used; these figures are not directly comparable across rows, since the datasets, splits, and evaluation protocols differ, and are listed only to confirm that the surveyed methods are strong, well-regarded detectors. The comparison this table draws is along the model-size (“Params”) and edge-validation columns. “N/R” denotes a value not reported in a directly comparable form in the cited work; “(F1)” marks an F1-score rather than accuracy.

Work	Architecture	Dataset	Params	Reported Acc.	Edge-Validated
RTIDS [19]	Transformer	CICIDS2017	N/R	99.17% (F1)	No
Yang et al. [21]	Vision Transformer	NSL-KDD	N/R	99.68%	No
FlowTransformer [20]	Transformer a	UNSW-NB15 a	varies a	∼0.97 (F1) a	No
Gueriani et al. (2024) [22]	CNN-LSTM	CICIoT2023	N/R	98.42%	No
Gueriani et al. (2025) [12]	LSTM-CNN-Attention	Edge-IIoTset	N/R	99.04%	No
This work (AutoML)	Compact CNN	CICIDS2017	1.18 M	91.22%	Yes (Pi 4)

^a FlowTransformer is a modular framework; the cited paper benchmarks many transformer configurations on NSL-KDD, UNSW-NB15 and CSE-CIC-IDS2018, so its parameter count and accuracy depend on the configuration. The value shown is the best macro-F1 reported on UNSW-NB15 (best F1 on CSE-CIC-IDS2018 is ≈0.985).

summarizes these approaches and contrasts them with this work on parameter count and edge validation. The accuracy figures listed in this table come from each cited paper’s own evaluation on a different benchmark and are not directly comparable across rows. Transferring the accuracy gains of transformer-based detectors onto an edge-deployable footprint is a natural direction for future work.

2.4. The IDS-ML Framework

The IDS-ML framework, selected for this study, integrates several techniques to improve detection accuracy:

• Ensemble Learning: It employs stacking and voting to combine outputs from base models like XGBoost [23] and LightGBM to reduce variance and improve robustness [1].
• Transfer Learning: The framework utilizes transfer learning to adapt pre-trained CNN models to intrusion detection tasks by converting network traffic into image-like data structures [1].
• HPO: The original authors utilized Bayesian Optimization to tune model parameters [7]. Our reproduction omits this tuning since it cannot alter the fixed parameter count of the pre-trained baseline architectures, which is the dimension of comparison most relevant to this work’s compression focus (see Section 3).

2.5. State-of-the-Art Enhancements

Despite the robustness of IDS-ML, the original authors noted that zero-day attack detection performance still has room for improvement [1]. Recent research supports this, with new frameworks emerging to address the “zero-touch” requirement of future networks. New studies demonstrate that AutoML can outperform manual tuning by jointly optimizing for accuracy and resource efficiency [5]. For example, recent surveys report that automated ensemble searches have achieved near-perfect F1-scores on benchmark datasets without human intervention [24]. Optimization for IDS spans hyperparameter tuning [7], feature selection, post-training model compression [14], and multi-objective formulations that jointly trade detection accuracy against resource cost [5]. The present work targets the neural architecture search branch of this space, using it specifically to obtain a model that fits within an edge-deployable size budget rather than to maximize accuracy in isolation.

3. Methodology

3.1. Dataset Description

We utilized the CICIDS2017 benchmark dataset for all experiments [4]. CICIDS2017 was selected for three reasons: it is the same benchmark used in the original IDS-ML framework [1], enabling direct comparability with the baseline results we reproduce in Phase 1; the raw PCAP captures are publicly available, which is required for the live replay benchmark in Phase 3; and it covers seven distinct attack classes spanning both signature-detectable (DoS, PortScan) and harder-to-detect (Infiltration, WebAttack) categories, providing a stress test across the attack spectrum.

3.2. Dataset Sampling and Preprocessing

The full CICIDS2017 dataset comprises over 2.8 million network traffic flows. Conducting AutoML hyperparameter searches over millions of rows introduces extreme computational overhead and frequently results in Out Of Memory (OOM) failures on standard GPU-accelerated environments. To mitigate this while preserving statistical validity, we utilized a stratified random sample provided by the authors of the original IDS-ML framework [1].

• Stratification Rationale: A stratified sampling approach preserves the class proportions of the original 2.8-million-row dataset. This ensures that rare attack vectors (such as Infiltration) remain proportionally represented, allowing for valid baseline comparisons and rapid architectural prototyping without the risk of hardware exhaustion.
• Data Cleaning: Infinite values (Infinity) and null entries (NaN) were stripped from the sample to ensure data integrity.
• Normalization: Feature scaling was applied using Min-Max normalization to bound the input vectors, which is critical for ensuring stable convergence during neural network gradient descent.
• Data Splitting: The stratified sample was further partitioned into training (80%) and testing (20%) sets. Stratification was maintained during this split to ensure the testing environment accurately reflected the training distribution.

3.3. Experimental Setup

The experiments were conducted using Python 3.8+ in a GPU-accelerated environment to support the training of deep learning models.

• Frameworks: The deep learning baselines (CNNs) were implemented using TensorFlow and Keras APIs. Scikit-learn [25] was utilized for data preprocessing and performance metric calculations.
• Evaluation Metrics: Model performance was evaluated using Accuracy and Weighted F1-Score. Model Complexity (number of trainable parameters) was also measured to assess computational efficiency.

This study follows a three-phase experimental approach utilizing the CICIDS2017 dataset [4].

3.4. Phase 1: Reproduction of Deep Learning Baselines

We selected the transfer learning component of the IDS-ML framework for reproduction [1]. This approach adapts pre-trained CNN image-classification models to the intrusion detection domain by reshaping tabular network traffic features into image-like data structures, a technique first demonstrated for malware traffic classification by Wang et al. [26]. As illustrated in Figure 1, the network flow features are scaled, zero-padded, and reshaped into $9\times 9$ single-channel images, which are then passed through pre-trained architectures via transfer learning. The implementation relies on TensorFlow/Keras for model construction and Scikit-learn [25] for data preprocessing and evaluation.

To establish a performance benchmark, we replicated five distinct deep learning CNN architectures adapted for IDS, as proposed in the IDS-ML framework [1]. The replicated models include:

• VGG16 and VGG19: Deep architectures utilizing small receptive fields.
• Xception: A model based on depth-wise separable convolutions.
• InceptionV3: An architecture designed for computational efficiency using factorized convolutions.
• InceptionResNetV2: A hybrid model combining Inception modules with residual connections.

All baseline models were trained using categorical cross-entropy loss and the Adam (Adaptive Moment Estimation) optimizer [27]. Adam was selected for its computational efficiency and its ability to adjust learning rates adaptively for each parameter. This adaptivity is particularly advantageous for the high-dimensional sparse data inherent in network traffic analysis. This ensures a stable convergence for complex architectures like InceptionResNet. We evaluated each model based on Accuracy and Weighted F1-score to determine the most effective baseline.

Each baseline architecture was trained with a fixed set of hyperparameters (Adam optimizer at default settings, batch size 32, 5 epochs, input images resized to 75 × 75). While the original IDS-ML framework applied Bayesian Optimization to tune these architectures [1], the objective of this study is model compression for edge deployment rather than peak baseline accuracy. Because the parameter count of each pre-trained architecture (e.g., 22.3 M for InceptionV3) is fixed by construction and independent of hyperparameter tuning, baseline HPO methodology does not affect the downstream finding that motivates this work: an AutoML-discovered compact architecture delivers comparable classification performance at a fraction of the model size, enabling operational deployment on resource-constrained hardware.

3.5. Phase 2: AutoML Optimization

To address the complexity of manual HPO identified in the original work [1], we introduce an AutoML Module.

• Rationale: Manual tuning is static and often fails to adapt to dynamic network environments. The original paper suggests that AutoML techniques should be deployed to realize automated intrusion detection [1].
• Implementation: We employed Keras Tuner’s Hyperband algorithm to perform automated neural architecture search over a bounded design space, simultaneously tuning architectural hyperparameters (number of convolutional blocks, filter counts, dense layer width, input resize dimensions, batch normalization, dropout) and training hyperparameters (learning rate). Preprocessing (Min-Max scaling, 9 × 9 reshape) and the use of a CNN backbone were held fixed. This focused search lets the algorithm trade architectural complexity directly against validation performance.

Following the baseline evaluation, we employed AutoML to synthesize an optimized model architecture focused on computational efficiency. The objective was to minimize the number of trainable parameters while maintaining classification accuracy comparable to the best-performing baseline, which was identified as InceptionV3 based on the Phase 1 replication results reported in Section 4.1. Given that the overarching objective of this study is to produce a model with high classification accuracy that is also suitable for deployment on resource-constrained edge hardware, subsequent phases carry forward only InceptionV3 rather than all five replicated architectures. Evaluating the AutoML model against the strongest baseline provides the most stringent benchmark: if the AutoML architecture matches or exceeds InceptionV3 on accuracy and F1, it necessarily exceeds the weaker baselines (VGG16, VGG19, Xception, and InceptionResNetV2), which were already outperformed by InceptionV3 in Phase 1. Including the lower-performing architectures in Phases 2 and 3 would therefore add computational overhead without changing the conclusions drawn from the comparison.

The AutoML process utilized Keras Tuner’s Hyperband algorithm, a bandit-based approach inspired by the multi-armed bandit problem, where computational resources are treated as a limited budget that must be allocated efficiently across competing candidate configurations [28]. The algorithm trains many candidates for a few epochs, progressively discards the worst performers, and allocates more epochs to the most promising candidates [7]. The search space included tunable hyperparameters such as image resize dimensions, number of convolutional blocks (1–3), filter counts, dense layer units, dropout rates, optional batch normalization, and learning rate. Early stopping on validation loss was used to prevent overfitting during the search. The final model was evaluated not only on predictive performance but also on model size (parameter count) to assess its suitability for deployment in resource-constrained environments.

The complete search space and Hyperband configuration are summarized in

Table 2. AutoML search space and hyperband configuration.

Hyperparameter	Type	Range/Choices
Searched (8 tunable hyperparameters)
img_size	Choice	{32, 48}
conv_blocks	Int	1–3
filters_i	Int	16–64, step 16 (per block)
batch_norm	Boolean	{True, False}
conv_dropout	Float	0.0–0.3, step 0.1
dense_units	Int	32–128, step 32
dense_dropout	Float	0.2–0.5, step 0.1
learning_rate	Choice	{${10}^{-3}$, ${10}^{-4}$}
Hyperband configuration
Objective	—	val_accuracy
Max epochs per trial	—	12
Reduction factor	—	3
Early stopping	—	monitor = val_loss, patience = 2
Class weights	—	balanced (per training distribution)
Final retrain epochs	—	10
Fixed (not searched)
Optimizer	—	Adam
Loss	—	Categorical cross-entropy
Kernel size	—	$3\times 3$, padding = same
Pooling	—	MaxPooling $2\times 2$
Hidden activation	—	ReLU
Output activation	—	Softmax

. The search optimized for validation accuracy with Adam as a fixed optimizer, categorical cross-entropy loss, 3 × 3 convolutional kernels with same-padding, 2 × 2 max-pooling after every convolutional block, ReLU activations on hidden layers, and softmax on the output layer. Class weights were computed from the training distribution and applied throughout the search to mitigate class imbalance. The best hyperparameter configuration identified by Hyperband was then retrained for 10 epochs to produce the final AutoML model evaluated in Phase 3.

3.6. Phase 3: Edge Deployment and Hardware Validation

The objective of this final experimental phase is explicitly distinct from the prior phases. While Phase 1 and Phase 2 empirically validated the classification efficacy (Accuracy, Precision, Recall, and F1-Score) of the models, Phase 3 evaluates the operational viability of these models in a physical environment. Specifically, we isolate and benchmark the computational overhead (inference latency, CPU utilization, and memory consumption) introduced by the InceptionV3 baseline versus the AutoML-optimized architecture when deployed as a Host-based Intrusion Detection System (HIDS).

3.6.1. Hardware Proxy Justification

The Raspberry Pi 4 was selected as a scientifically valid hardware stand-in because its System on a Chip (SoC), based on the ARM Cortex A72, mirrors the computational ceilings found in industry-standard commercial IoT gateways such as the Siemens SIMATIC IoT2050 [29]. The specific hardware constraints of the experimental deployment are detailed in

Table 3. Edge testbed hardware specifications for Sniffer/IDS node (Raspberry Pi 4 Model B).

Component	Specification
Processor (CPU)	Broadcom BCM2711, Quad-core Cortex A72 (ARM v8) 64-bit
Clock Speed	1.5 GHz
Memory (RAM)	2 GB LPDDR4-3200 SDRAM
Storage	32 GB MicroSDHC (Class 10)
Network Interface	Gigabit Ethernet, 2.4 GHz and 5.0 GHz IEEE 802.11ac wireless
Operating System	Raspbian GNU/Linux 12 (Bookworm), 64-bit kernel (aarch64) running 32-bit armhf userspace, Kernel 6.12.75

.

3.6.2. Network Isolation and Out-of-Band (OOB) Management

To ensure the empirical integrity of the hardware benchmarks and prevent traffic pollution, the testbed utilized a strict Out-of-Band (OOB) management strategy. In network security evaluations, using a single interface for both administrative control and packet sniffing introduces an observer effect, where the telemetry and SSH control traffic artificially inflate the device’s CPU utilization and inject anomalous data into the classification engine.

To mitigate this, as explicitly detailed in Figure 2, the Raspberry Pi’s Gigabit Ethernet interface was dedicated entirely to packet ingestion. The interface was connected directly to a managed switch configured with port mirroring (SPAN) [30], allowing it to operate as a passive, dedicated sniffer. All administrative commands, process monitoring, and metric extractions were segregated and routed out-of-bounds via the device’s IEEE 802.11ac wireless interface. This physical bifurcation guarantees that the measured inference latency and memory overhead are exclusively attributable to the AutoML model processing the target network traffic, yielding highly accurate and uncontaminated benchmarks.

The testbed architecture is presented across two complementary figures: Figure 2 shows the physical network topology of the lab experiment, while Figure 3 details the onboard inference pipeline running on the Sniffer/IDS node.

Figure 2 depicts the full testbed topology. The Attacker node (Raspberry Pi 3) stores the CICIDS2017 PCAP captures on a locally attached USB hard drive and replays them via tcpreplay over its dedicated Ethernet interface. This node constitutes the active threat source in the testbed: each replayed PCAP reproduces a specific day of attack traffic from the original CICIDS2017 collection (DoS, BruteForce, Botnet, Infiltration, PortScan, and WebAttack campaigns), subjecting the Sniffer to the full diversity of attack classes covered by the dataset. The managed Ethernet switch is configured with port mirroring (SPAN), forwarding all traffic from the Attacker’s port to the Sniffer’s capture port. The Sniffer node (Raspberry Pi 4) simulates the resource-constrained edge device that hosts the IDS, performing all packet capture, flow-level feature extraction, and TFLite model inference locally without offload. The host workstation shown in the figure serves exclusively as the administrative control plane and is not part of the IDS data path. Both Raspberry Pi devices operate headless, without attached monitors or keyboards, as is standard for production edge deployments; the host workstation is therefore required to initiate test runs, configure model selection, and retrieve benchmarking telemetry via the lightweight web interfaces hosted on each Pi. The host connects only via the wireless management segment, never touches the Ethernet data path, and plays no role in the inference or classification pipeline; its removal after test initiation would not affect any measurement reported in this study.

Figure 3 details the onboard inference process on the Sniffer node (Raspberry Pi 4). Mirrored Ethernet traffic is ingested via scapy and processed through a six-stage pipeline: packet capture, flow-level feature extraction (77 CICIDS2017 features), MinMax normalization and reshaping to a $9\times 9\times 1$ tensor, TFLite inference using the currently selected model (AutoML or InceptionV3), seven-class softmax classification, and live dashboard logging via Flask. All intrusion classification is performed on device without cloud offload or external compute. A System Monitor process co-located on the Sniffer samples CPU utilization, memory consumption, and die temperature at one-second intervals throughout each run, providing the hardware telemetry reported in Section 4. The physical and functional separation between the Ethernet data plane (Figure 2) and the wireless control plane guarantees that all performance measurements reflect the intrinsic cost of edge-native IDS inference rather than the overhead of administrative traffic or shared network resources.

3.6.3. TensorFlow Lite Model Conversion

Standard deep learning models (saved as .keras or .h5 files) carry significant framework overhead designed to support backpropagation and distributed training. This overhead is unnecessary for inference-only deployment on memory-constrained edge devices [16]. To bridge this gap, the optimized AutoML model was converted into a TensorFlow Lite (.tflite) flat-buffer format. Because the full TensorFlow framework is not available for the Raspberry Pi’s 32-bit ARM (armhf) userspace, the conversion was performed on a host machine using a Docker containerized Python 3.11 environment with TensorFlow installed, and the resulting .tflite artifacts were transferred to the edge device. This conversion utilized standard 32-bit floating-point (FP32) preservation rather than destructive integer quantization. Preservation of FP32 was selected over integer quantization (INT8) to maintain a controlled experimental comparison. Integer quantization reduces model size and can improve inference speed, but introduces accuracy degradation that typically requires quantization-aware retraining to mitigate [15]. By preserving the same mathematical precision used during training, any performance differences observed in Phase 3 are attributable to hardware constraints alone, not to changes in model precision. Integer quantization is reserved as a future optimization pathway. The neural network weights and structural integrity therefore remain numerically equivalent to the original cloud-trained model up to the small floating-point differences introduced by TFLite’s op fusion and kernel ordering. This guarantees that conversion to .tflite explicitly does not reduce or alter the classification accuracy established in Phase 2; it only optimizes the graph execution pathways and memory allocation for ARM-based processors [16].

3.6.4. TFLite Runtime Configuration on the Edge Device

On the Raspberry Pi 4, both models were executed using the standalone tflite-runtime 2.14.0 Python package rather than the full TensorFlow library, which reduces import overhead and runtime memory footprint. The TFLite Interpreter was instantiated with default settings (no explicit num_threads parameter), which results in single-threaded CPU inference on the Cortex-A72; the remaining three cores remained available to scapy-based packet capture and the Flask telemetry server. The XNNPACK delegate was not enabled; standard built-in TFLite kernels were used for all operations. Each inference call is synchronous and blocking, executing in order: feature scaling via the loaded scaler.pkl, zero-padding from 77 to 81 features, reshaping to a $9\times 9\times 1$ tensor, set_tensor(), invoke(), and get_tensor(). The interpreter instance is cached after first load so subsequent inferences reuse the allocated tensors. Both models used an identical runtime configuration so that the relative comparisons reported in Section 4 reflect only differences in model architecture and not differences in runtime tuning.

3.6.5. Preservation of the Preprocessing State

A common cause of silent model failure during edge deployment is “training serving skew,” wherein the statistical parameters used to scale data during training do not perfectly match the parameters used during live inference [31]. To ensure determinism, the preprocessing pipeline states were serialized and migrated to the edge device alongside the model, as illustrated in Figure 4.

• scaler.pkl: Contains the serialized Min-Max normalization boundaries extracted from the training distribution. This guarantees that real-time raw packet features are scaled using the exact mathematical bounds the model was trained to recognize, preventing concept drift.
• encoder.pkl: Contains the target label dictionary, ensuring the raw array output of the final dense layer is mapped to the correct attack classification strings (e.g., DoS, Bot, PortScan).

Because the same scaler.pkl and encoder.pkl artifacts were fit once during data preprocessing and reused across both Phase 1 and Phase 2 training, a single serialized preprocessing state suffices to support inference for both the InceptionV3 baseline and the AutoML model on the edge device. By migrating these shared artifacts alongside the converted .tflite models, the preprocessing pipeline used during training was preserved on the Raspberry Pi 4. To validate this, the same 11,316-sample test dataset used during Phase 2 model evaluation was executed directly through both TFLite models (InceptionV3 and AutoML) on the Pi 4. Both models produced classification outputs identical to their Phase 2 counterparts, confirming the absence of training serving skew and establishing a controlled baseline for the hardware performance benchmarks conducted in Test 3.

4. Results

This section presents empirical results from the three experimental phases introduced in Section 3. Section 4 first reports the Phase 1 reproduction of the five deep learning baselines on the CICIDS2017 dataset, identifying the top-performing architecture against which the AutoML enhancement is compared. The Phase 2 AutoML enhancement results follow, quantifying the reduction in model complexity achieved while preserving classification performance, together with per-class detection metrics and training convergence behavior. The section concludes with the Phase 3 edge deployment results, which validate TFLite conversion integrity, measure inference latency on isolated test data, and benchmark full system performance (CPU, memory, thermal, and throughput) under sustained PCAP replay on the Raspberry Pi 4.

4.1. Phase 1: Reproduction Results

We first replicated the model training and testing against the CICIDS2017 dataset using five deep learning architectures: VGG16, VGG19, Xception, InceptionV3, and InceptionResNetV2 [1]. As shown in Figure 5 and

Table 4. Comparison of replicated deep learning architectures on CICIDS2017.

Model	Accuracy	Weighted F1	Parameters
VGG16	0.7695	0.7797	14,847,845
VGG19	0.7095	0.7412	20,157,541
Xception	0.8784	0.8828	21,387,853
InceptionV3	0.9153	0.9186	22,329,157
InceptionResNetV2	0.8624	0.8660	54,732,037

, our replication identified InceptionV3 as the highest-performing baseline model with an accuracy of 91.53% and a Weighted F1-Score of 0.9186. This result is consistent with the original IDS-ML framework [1], in which InceptionV3 was also identified as the top-performing CNN baseline among the five replicated architectures. Its parameter-efficient architectural design [32] offered the best accuracy–complexity trade-off among the replicated baselines, outperforming the substantially larger InceptionResNetV2 (54.7 M parameters) and the simpler VGG variants in both accuracy and F1. Building on this baseline, we introduced an optimized model generated via AutoML. As demonstrated in Section 4.2, the AutoML model achieved an accuracy of 91.22% and a Weighted F1-Score of 0.9203, matching the InceptionV3 baseline on F1 while achieving a 94.7% reduction in model complexity, decreasing the parameter count from 22.3 million to just 1.18 million, improving computational efficiency for IDS deployment.

4.2. Phase 2: AutoML Enhancement Results

The proposed AutoML enhancement was evaluated against the manually tuned baseline to assess both predictive performance and computational efficiency. As detailed in

Table 5. AutoML enhancement performance by class on CICIDS2017 test set.

Class	Precision	Recall	F1-Score	Support
BENIGN	0.9825	0.8140	0.8904	4544
Bot	0.6650	0.9949	0.7971	391
BruteForce	0.6850	0.9910	0.8101	553
DoS	0.9653	0.9739	0.9696	3797
Infiltration	0.0312	0.8571	0.0603	7
PortScan	0.9499	0.9918	0.9704	1588
WebAttack	0.8392	0.9335	0.8838	436
Overall Accuracy	0.9122			11,316

, the AutoML model achieved an overall accuracy of 0.9122 and a Weighted F1-Score of 0.9203, statistically on par with the InceptionV3 baseline’s 0.9186 (the small difference is not significant; see Section 4.2.1), with consistent detection across major attack vectors and high F1-scores for critical categories such as DoS (0.9696) and PortScan (0.9704).

The class-wise performance is further detailed in

Table 6. Detailed confusion metrics by class (derived from confusion matrix). The FP and FN column totals are necessarily equal in single-label multi-class classification: each misclassified sample contributes one false negative to its true class and one false positive to its predicted class.

Class	Support	TP	FP	FN	Detection
	(Total)	(Correct)	(False Alarm)	(Missed)	Rate (Recall)
BENIGN	4544	3699	66	845	81.40%
Bot	391	389	196	2	99.49%
BruteForce	553	548	252	5	99.10%
DoS	3797	3698	133	99	97.39%
Infiltration	7	6	186	1	85.71%
PortScan	1588	1575	83	13	99.18%
WebAttack	436	407	78	29	93.35%
Totals	11,316	10,322	994	994	–

and visualized in Figure 6, which explicitly quantifies the True Positive and False Positive counts. The model maintained exceptionally high recall for critical threats such as Bot (0.9949) and BruteForce (0.9910), missing only 2 and 5 attacks, respectively, out of the test set. The model demonstrated lower precision within the Infiltration class (precision 0.0312, with 6 true positives against 186 false positives). This is directly attributable to the class’s extremely low support (only 7 instances in the test set, approximately 0.06% of the data) which is a known characteristic of the CICIDS2017 dataset [4] that inherently limits the ability of deep learning models to learn discriminative feature representations for ultra-rare attack vectors. Standard mitigation includes cost-sensitive learning, focal loss [33], oversampling techniques such as SMOTE, and architectural alternatives such as a dedicated one-class anomaly head for rare classes; none of these are explored in this work, but they are natural avenues for future improvement when the deployment scenario specifically targets ultra-rare attack types.

Table 7. Binary classification view of the AutoML model (BENIGN vs. Attack), derived directly from Table 6 by collapsing the six attack classes into a single positive class.

	Predicted: Attack	Predicted: BENIGN	Total
Actual: Attack	6706 (TP)	66 (FN)	6772
Actual: BENIGN	845 (FP)	3699 (TN)	4544
Total	7551	3765	11,316
Attack class metrics		Benign class metrics
Precision	0.8881	Specificity (TNR)	0.8140
Recall (TPR)	0.9903	False Positive Rate	0.1860
F1-Score	0.9365	False Negative Rate	0.0097
Overall Binary Accuracy: 0.9195

provides a binary view (BENIGN vs. Attack) of the same predictions. Two numbers characterize the model’s operating point: a false negative rate of 0.97% and a false positive rate of 18.6%. The model misses almost no attacks but raises false alarms on roughly one in five benign flows. For an intrusion detection system, this is the safer direction in which to err. A false negative, that is an undetected intrusion, can lead to data exfiltration, ransomware deployment, or lateral movement before a defender is aware of it, whereas a false positive costs an analyst the time to triage and dismiss an alert. An IDS that errs toward raising too many alerts is therefore generally preferable to one that errs toward silent failure. The class imbalance in CICIDS2017 [4] pushes the trained model in this safer direction.

The Infiltration class is where this asymmetry is most extreme. With only 7 Infiltration samples in the test set (approximately 0.06% of the data), the model cannot learn a precise decision boundary: 186 of 192 predicted Infiltration alerts are false positives, while still recovering 6 of the 7 true Infiltration flows. The detection objective is met, since the attacks are flagged, but the alerts are too noisy to act on directly. A deployment targeting this attack type would either raise the Infiltration decision threshold, trading recall for precision, or treat Infiltration alerts as candidate anomalies pending corroboration from additional signals.

The primary advantage of the AutoML approach is demonstrated in

Table 8. Efficiency comparison: replicated baseline vs. AutoML enhancement.

Model	Accuracy	Weighted F1	Parameters	Size Reduction
Replication (InceptionV3)	0.9153	0.9186	22,329,157	—
AutoML (Ours)	0.9122	0.9203	1,180,615	−94.7%

and visualized in Figure 7. While the InceptionV3 baseline required over 22 million parameters, the AutoML optimization synthesized a model with only 1,180,615 parameters. This equates to a 94.7% reduction in model size, substantially lowering the computational memory and storage requirements, while matching the classification performance of the strongest baseline. As the statistical comparison below shows, the differences in accuracy and Weighted F1 between the two models are not statistically significant.

4.2.1. Statistical Comparison with the InceptionV3 Baseline

To go beyond a single-point comparison, the AutoML and InceptionV3 models were compared statistically on the same 11,316-sample test set, using their existing predictions rather than retraining. Both analyses below treat classification quality as the only object of comparison; the model-size and latency results are reported separately in Table 8 and Section 4.

McNemar’s test [34] was applied to the paired predictions. Of the 11,316 test samples, both models classified 10,101 correctly and 737 incorrectly; the AutoML model alone was correct on 221, and the InceptionV3 model alone on 257. With these 478 discordant pairs, the exact (binomial) two-sided p-value is $0.11$ (the ${\chi }^2$ statistic with continuity correction is $2.56$, $p=0.11$), so the hypothesis that the two models have the same error rate is not rejected at the $0.05$ level. In addition, 2000 bootstrap resamples of the test predictions give 95% confidence intervals of $[0.9071, 0.9171]$ (AutoML) and $[0.9103, 0.9203]$ (InceptionV3) for accuracy, and $[0.9155, 0.9249]$ (AutoML) and $[0.9137, 0.9233]$ (InceptionV3) for Weighted F1 [35]; the intervals overlap heavily, and the bootstrap confidence interval for the difference (AutoML minus InceptionV3) includes zero for both metrics, at $[-0.0071,+0.0008]$ for accuracy and $[-0.0018,+0.0053]$ for Weighted F1. These figures are summarized in

Table 9. Statistical comparison of the AutoML and InceptionV3 models on the 11,316-sample CICIDS2017 test set, computed from the existing test predictions (no retraining). Brackets give bootstrap 95% confidence intervals ($B=2000$ resamples).

Metric	AutoML	InceptionV3
Accuracy	0.9122  $[0.9071,0.9171]$	0.9153  $[0.9103,0.9203]$
Weighted F1	0.9203  $[0.9155,0.9249]$	0.9186  $[0.9137,0.9233]$
$\Delta$ Accuracy (AutoML − InceptionV3): $-0.0032$, 95% CI $[-0.0071,+0.0008]$
$\Delta$ Weighted F1 (AutoML − InceptionV3): $+0.0017$, 95% CI $[-0.0018,+0.0053]$
McNemar’s test on paired predictions: $b=221$, $c=257$ discordant; exact two-sided $p=0.11$

.

The two models are therefore statistically indistinguishable in classification quality on this benchmark: the $0.9203$ versus $0.9186$ Weighted-F1 gap is within sampling noise. The contribution of the AutoML model is not that it is more accurate than InceptionV3, but that it reaches the same accuracy with $94.7\%$ fewer parameters, together with the latency, CPU, and memory advantages reported in Section 4. A fuller study with repeated training runs and multiple random seeds, which would place confidence bands on the training process itself and not only on the test predictions, is left to future work.

4.2.2. Training Convergence

To visually confirm that the 94.7%-compressed AutoML architecture did not suffer from internal overfitting during the training phase, the learning curves were plotted across all epochs.

As illustrated in Figure 8, both the training and validation loss converge smoothly and in close alignment, plateauing without significant divergence. The validation loss decreases steadily alongside the training loss, indicating healthy gradient descent without the characteristic sudden spike that denotes memorization. The small gap between the final training and validation loss indicates that the model generalized to the isolated 20% internal hold out set without overfitting.

4.3. Phase 3: Edge Deployment Results

4.3.1. TFLite Conversion Integrity

A prerequisite to benchmarking hardware performance is confirming that the model conversion process itself does not degrade classification accuracy. As described in Section 3, both the InceptionV3 baseline and the AutoML model were converted from the native Keras (.keras) format into the TensorFlow Lite (.tflite) flatbuffer format using standard 32-bit floating-point (FP32) preservation [16].

To empirically verify this claim, the identical 11,316 sample stratified test set used in Phase 1 was executed directly through both TFLite models on the Raspberry Pi 4 edge device. The pre-scaled feature vectors were fed directly to the TFLite interpreter, bypassing any network capture or real-time feature extraction pipeline. This isolates the conversion process as the sole experimental variable, ensuring that any observed accuracy deviation is exclusively attributable to differences between the TensorFlow and TFLite inference engines.

As demonstrated in

Table 10. TFLite conversion integrity: Colab Keras vs. Raspberry Pi 4 TFLite.

Model	Env.	Accuracy	Weighted F1	Conversion Loss
AutoML	Colab (Keras)	0.9122	0.9203	—
AutoML	Pi 4 (TFLite)	0.9122	0.9203	0.00%
InceptionV3	Colab (Keras)	0.9153	0.9186	—
InceptionV3	Pi 4 (TFLite)	0.9153	0.9186	0.00%

, both models produced identical predicted classes for all 11,316 test samples relative to their Keras counterparts; the underlying logits may differ at the floating point level due to TFLite’s op fusion and kernel ordering, but these differences are below the threshold required to change any classification decision. The AutoML model maintained its accuracy of 91.22% and Weighted F1-Score of 0.9203, while the InceptionV3 baseline preserved its 91.53% accuracy and 0.9186 F1-Score. This result confirms that the TFLite conversion preserves the neural network weights and execution behavior on ARM processors [16], validating that any performance metrics observed during hardware benchmarking reflect the true model behavior without conversion-induced artifacts [18].

Table 11. Per-class conversion validation: AutoML Keras (Colab) vs. TFLite (Pi 4).

Class	Keras Precision	TFLite Precision	Keras Recall	TFLite Recall	Keras F1	TFLite F1
BENIGN	0.9825	0.9825	0.8140	0.8140	0.8904	0.8904
Bot	0.6650	0.6650	0.9949	0.9949	0.7971	0.7971
BruteForce	0.6850	0.6850	0.9910	0.9910	0.8101	0.8101
DoS	0.9653	0.9653	0.9739	0.9739	0.9696	0.9696
Infiltration	0.0312	0.0312	0.8571	0.8571	0.0603	0.0603
PortScan	0.9499	0.9499	0.9918	0.9918	0.9704	0.9704
WebAttack	0.8392	0.8392	0.9335	0.9335	0.8838	0.8838

provides a per-class validation for the AutoML model, confirming that the conversion preserves classification behavior at every decision boundary. The Precision, Recall, and F1-Score for each attack class are identical between the Colab Keras environment and the Raspberry Pi 4 TFLite deployment, with differences attributable solely to display rounding.

4.3.2. Inference Latency on Edge Hardware (Isolated Model Inference)

With conversion integrity established, inference latency was measured for both TFLite models on the Raspberry Pi 4 during the conversion validation test. This measurement isolates the cost of the TFLite model itself: the 11,316 pre-scaled feature vectors were fed directly to the interpreter, bypassing live packet capture, flow tracking, and feature extraction. The latency reported here therefore represents pure model inference on edge hardware, distinct from the end-to-end pipeline latency measured later in Test 3 (reported in Section 4.3.4 below), which additionally includes scapy-based packet capture and on-the-fly flow feature extraction overhead. As shown in

Table 12. Isolated-inference latency comparison on Raspberry Pi 4 (11,316 pre-scaled test samples; capture and feature-extraction overhead excluded).

Model	Avg (ms)	P95 (ms)	Min (ms)	Max (ms)
AutoML	3.00	3.20	2.90	5.39
InceptionV3	84.71	88.42	78.44	507.26
Speedup	28×	28×	27×	94×

, the AutoML model achieved an average inference time of 3.00 ms per classification, compared to 84.71 ms for the InceptionV3 baseline, a 28× speedup. This result directly validates the operational advantage of the 94.7% parameter reduction established in Phase 2. The compressed AutoML architecture not only matches the baseline on accuracy but executes inference approximately 28 times faster on identical hardware.

The P95 latency (3.20 ms vs. 88.42 ms) confirms that this speedup is consistent and not driven by outliers. The InceptionV3 model exhibited a maximum inference spike of 507.26 ms, likely attributable to ARM memory management overhead when processing the 85 MB model, whereas the AutoML model’s maximum latency remained below 6 ms. For real-time intrusion detection, where classification must keep pace with network traffic throughput, this latency differential is significant [17].

4.3.3. Hardware Performance Profiling (Test 3)

To quantify the full computational impact of sustained model inference under realistic network loads, both models were deployed for continuous classification against the complete CICIDS2017 PCAP dataset replayed over the isolated Ethernet testbed (Figure 2). The CICIDS2017 dataset comprises five daily PCAP captures totaling approximately 50 GB of raw network traffic. The PCAP files were obtained directly from the CICIDS2017 dataset repository [4] and comprise five daily captures (Monday through Friday), encompassing the full range of benign traffic and attack scenarios present in the dataset. These PCAP captures are the raw network traffic from which the CICIDS2017 CSV flow features used in Phases 1 and 2 were originally derived using CICFlowMeter [4]. All five daily captures were replayed sequentially using tcpreplay at original capture timing over the dedicated Ethernet interface, while the Raspberry Pi 4 performed real-time packet capture, flow-level feature extraction, and TFLite model inference on the ingested traffic.

System-level and process-level performance metrics were sampled at one-second intervals for the duration of each model’s complete test run. Specifically, the following metrics were instrumented:

• CPU Utilization: Total system CPU percentage and the IDS model process CPU percentage were recorded independently, enabling isolation of the model’s computational footprint from operating system overhead.
• Memory Consumption: System-wide and process-specific RAM usage were tracked to quantify the memory overhead introduced by each model architecture.
• Inference Latency: Per classification inference time was logged for every flow classification event, enabling computation of average, P95, minimum, and maximum latency under sustained load.
• Thermal Behavior: CPU die temperature was monitored to assess whether sustained inference induces thermal throttling on the passively cooled ARM processor.
• Throughput: Network ingestion rate (packets per second) and classification throughput (inferences per second) were measured to determine the maximum traffic volume each model can sustain without dropping flows.

This instrumentation enables a direct comparison of the operational cost of deploying a 22.3 million parameter baseline (InceptionV3) versus a 1.18 million parameter AutoML-optimized model on identical constrained hardware, addressing a critical gap in the IDS literature where models are frequently evaluated on classification accuracy alone without consideration of deployment feasibility [36,37].

4.3.4. Hardware Performance Results

Both models were evaluated over the complete CICIDS2017 PCAP dataset, with each test run spanning approximately 41 h of continuous inference.

Table 13. Hardware performance comparison: AutoML vs. InceptionV3 on Raspberry Pi 4 (Test 3). Note: Unlike the other tables in this paper, this table lists metrics as rows and models as columns due to the larger number of metrics being reported.

Metric	AutoML	InceptionV3	Ratio
Model Size	4.5 MB	85 MB	19× smaller
Avg Inference Latency	4.76 ms	89.03 ms	18.7× faster
P95 Inference Latency	7.71 ms	93.51 ms	12.1× faster
Max Inference Latency	10.86 ms	147.08 ms	13.5× faster
Inferences/s	10.4	5.81	1.8× higher
Avg System CPU	10.2%	34.8%	3.4× lower
Avg Model CPU	10.0%	34.6%	3.5× lower
Peak System CPU	49.0%	69.7%	—
Peak Model CPU	32.0%	67.2%	—
Avg System Memory	25.2% (465 MB)	31.3% (578 MB)	—
Avg Model Memory	9.2% (170 MB)	19.1% (353 MB)	2.1× lower
Peak System Memory	27.7% (511 MB)	33.2% (613 MB)	—
Peak Model Memory	10.5% (194 MB)	20.2% (373 MB)	—
Total Inferences	1,542,926	862,194	1.8× more
Run Time	41.2 h	41.2 h	—

presents the aggregate performance comparison.

As shown in Figure 9, the AutoML model achieved an average inference latency of 4.76 ms under sustained load, compared to 89.03 ms for InceptionV3, representing an 18.7× speedup. These values are higher than the isolated-inference figures in Table 12 (3.00 ms and 84.71 ms, respectively) because the Test 3 pipeline additionally measures live packet capture, flow tracking, and on-the-fly feature extraction. The difference quantifies the per-flow overhead of the end-to-end IDS pipeline beyond pure model inference. The P95 latency (7.71 ms vs. 93.51 ms) confirms this advantage is consistent across the latency distribution. Over the 41 h test, the AutoML model completed 1,542,926 inferences compared to InceptionV3’s 862,194, yielding a 1.8× throughput advantage. This disparity is a direct consequence of the inference latency differential: because InceptionV3 requires 89 ms per classification versus AutoML’s 4.76 ms, the larger model’s inference pipeline occupies the CPU for longer periods, reducing the number of flows that can be classified before they expire due to inactivity timeouts. Under identical traffic conditions, the AutoML architecture therefore provides greater classification coverage of network activity.

CPU and Memory Utilization

Figure 10 and Figure 11 illustrate the CPU and memory overhead of each model. The AutoML model consumed an average of 10.0% CPU (process level) with peaks reaching 32%. In contrast, InceptionV3 consumed 34.6% CPU on average, with peaks reaching 67.2%. This 3.5× difference in CPU overhead directly impacts the feasibility of co-hosting additional services (e.g., IoT-device-specific software or firmware) alongside the IDS on resource-constrained hardware [17].

Memory consumption followed a similar pattern: AutoML required 9.2% of system RAM (approximately 170 MB) compared to InceptionV3’s 19.1% (approximately 353 MB). As shown in Figure 12, both models exhibited stable memory profiles throughout the 41 h test, with no evidence of memory leaks or unbounded growth. This stability is expected because TFLite inference is a read-only operation on fixed model weights, and the supporting data structures (flow tables, metric buffers, and classification logs) are bounded in size. Temporary allocations for per-inference feature arrays are released after each classification cycle, preventing memory accumulation over extended operation.

Temporal Behavior

Figure 13 and Figure 14 present the CPU utilization and thermal behavior over the full test duration. The CPU time series reveals that InceptionV3 sustains consistently higher processor load throughout the run, with frequent peaks approaching 70%. The AutoML model maintains a lower and more uniform CPU profile.

CPU die temperature, monitored on the passively cooled Raspberry Pi 4, showed that InceptionV3 induced higher sustained thermal load. While neither model caused thermal throttling during the test period, the temperature differential indicates that InceptionV3 would have reduced thermal headroom in warmer ambient environments or enclosed industrial deployments [38].

5. Discussion

5.1. Significance of Model Compression and Regularization

The most critical finding of this study is the 94.7% reduction in model parameters (from 22.3 M to 1.18 M) achieved by the AutoML-generated model. In the context of IDS, this reduction is operationally significant for two major reasons:

• Edge Deployment: Modern networks increasingly rely on Edge and IoT devices to filter traffic locally. These devices often lack the memory to host heavy architectures like InceptionV3 [17,38]. Our compact model enables deployment directly on resource-constrained gateways without relying on cloud offloading, addressing the challenge of identifying attacks in industrial IoT environments [39].
• Architectural Regularization: As established in the foundational deep learning literature, massive baseline architectures possess excess capacity that inherently increases the risk of memorizing training set artifacts, leading to overfitting [40]. By compressing the model footprint to approximately 1.18 million parameters, the AutoML architecture may act as a form of implicit regularization through reduced model capacity, a known effect in deep learning where smaller architectures are less prone to memorizing training set artifacts [41,42].

5.2. The AutoML Advantage

AutoML fundamentally shifts the model development paradigm from manual trial and error to algorithmic optimization. As noted in the IDS-ML framework, manually tuning hyperparameters is time consuming and often fails to capture complex interactions between model depth and learning rates [7].

The AutoML process employed in this study, illustrated in Figure 15, operates as a continuous feedback loop using Keras Tuner’s Hyperband algorithm:

1.

Selection: Instead of random guessing, the Hyperband algorithm uses an adaptive resource allocation strategy inspired by the successive halving bandit method. It trains a large number of candidate configurations for a small number of epochs, then progressively discards the worst performers and allocates more epochs to the most promising candidates.

2.

Evaluation and Update: As shown in the diagram, each candidate model is trained and evaluated against specific metrics (Accuracy and Parameter Count). Underperforming candidates are eliminated early, focusing computational resources on the configurations most likely to yield optimal results [7].

3.

Convergence: This cycle repeats across multiple Hyperband brackets until the objective function balancing high F1 scores with minimal size is satisfied, resulting in the optimized architecture presented in our results [1].

This automated approach allowed us to discover a compact two-block CNN architecture (1.18 M parameters) that is substantially more efficient than the larger pre-trained baselines, supporting the use of AutoML for future autonomous defense systems. This makes our approach complementary to the transformer-based and attention-based IDS architectures reviewed in Section 2 rather than a competitor to them. Those methods aim to maximize detection accuracy on benchmark datasets, whereas the neural architecture search used here aims to minimize model size so that a usable detector runs on constrained edge hardware.

5.3. Edge Deployment Implications

5.3.1. Operational Viability on Constrained Hardware

The Phase 3 hardware benchmarks confirm that the AutoML model is operationally viable for continuous IDS deployment on resource-constrained edge devices. With an average CPU load of 10.0% and memory footprint of 170 MB, the model leaves substantial headroom for co-hosting additional services such as logging, alerting, or device management on the same hardware. In contrast, InceptionV3’s 34.6% average CPU and 353 MB memory consumption approach the practical limits of the Raspberry Pi 4, limiting its suitability for multi-service edge gateways.

The 18.7× inference speedup translates into greater classification coverage of network activity: AutoML completed 1.8× more inferences than InceptionV3 over the 41 h run because the lower per-flow latency leaves more flows classified before they expire due to inactivity timeouts, rather than dropped while the heavier model is still occupying the CPU. As illustrated in Figure 16, this architecture enables OT and industrial IoT devices to operate their own HIDS, which is particularly valuable where network-level inspection (NIDS) may be blinded by end-to-end encryption [17].

5.3.2. Applicability to OT and Industrial IoT

The capability to embed lightweight models is especially valuable for Operational Technology (OT) and Industrial IoT (IIoT). Unlike IT networks, OT systems often run on legacy hardware where frequent software updates are operationally risky; a failed update can cause costly production downtime or compromise physical safety [38].

Traditional signature-based IDSs require constant database updates to recognize new threats, making them ill suited for these “set and forget” industrial environments [4]. In contrast, our AutoML-generated model learns feature representations directly from traffic patterns rather than relying on fixed signatures. This eliminates the operational dependency on continuous signature database updates which is a practical advantage for OT and IIoT deployments where downtime windows for updates are limited or unavailable. The closed set evaluation in this work does not directly demonstrate zero-day detection; rather, it shows that the compressed model retains the same generalization properties as the InceptionV3 baseline. Formal zero-day evaluation, including held out attack family experiments, is reserved for future work (Section 6).

5.4. Threats to Validity and Limitations

This study has several limitations that should be considered when interpreting the results.

Reported accuracy and F1 figures come from a single training run on a single 80/20 stratified split. The 0.0017 F1 difference between AutoML and InceptionV3 falls within the run-to-run variance typical for CNN training on CICIDS2017 and should not be read as a definitive accuracy ranking. The compression and latency findings (94.7% smaller, 18.7× faster) are not affected by this issue, since parameter counts and inference FLOPs are deterministic properties of the trained network rather than statistical estimates.

The Phase 1 baselines were trained for 5 epochs with default Adam hyperparameters, while the AutoML model was selected by Hyperband (up to 12 epochs per trial) and the chosen architecture was then retrained for 10 epochs; the per-model training budgets are therefore short and broadly comparable, though not matched exactly, and what differs is the search budget, since Hyperband evaluates many candidate configurations. The compression result still holds because each pre-trained architecture has a fixed parameter count regardless of how it is trained, but a like-for-like baseline tuning pass would tighten the accuracy comparison and is left for future work.

No standard compression baselines (INT8 quantization, weight pruning, or knowledge distillation) were evaluated as alternatives to AutoML. The paper uses FP32 TFLite throughout in order to keep model architecture as the only variable in the size and latency comparison. Quantization was deferred (Section 3) and would compound the AutoML reduction rather than replace it; a quantized AutoML model would be smaller still.

The Hyperband search space was restricted to small architectures: 1–3 convolutional blocks, 16–64 filters per block, 32–128 dense units, and an input resize of 32 or 48. The 94.7% reduction therefore reflects a search bounded by what can plausibly run on a Raspberry Pi 4. The aim of the search was the best architecture inside the edge-deployable envelope, not the unconstrained accuracy maximum.

The evaluation is closed set: the same seven CICIDS2017 attack classes appear in both training and test data, and no held-out attack family is reserved to probe true zero-day generalization. This study targets the model compression and edge deployment axes; a leave one class out experiment or evaluation against a separate novel attack dataset belongs to future work (Section 6). The experiments are also confined to a single benchmark; CICIDS2017 was chosen for direct comparability with the reproduced IDS-ML baselines and for the public availability of the raw PCAP captures required by the Phase 3 replay test (Section 3). Because this is a single-dataset study, it does not by itself establish generalization. Cross-dataset and cross-domain validation including on the OT and IIoT benchmarks named in Section 6 is a prerequisite for any generalization claim and is left to future work.

Subsequent analysis by Engelen et al. [43] has documented systematic problems in CICIDS2017 and in the CICFlowMeter tool used to build it, spanning traffic generation, flow construction, feature extraction, and labeling. Examples include TCP flag count features that are only ever zero or one because only the first packet of a flow is inspected, a duplicated Fwd Header Length column, and a flow construction bug in which a TCP connection is not terminated on its FIN handshake, so that adjacent connections are merged or assigned the wrong direction and flows near attack time window boundaries are mislabeled. The authors reconstruct and relabel more than 20% of the original traces and release a corrected dataset together with a fixed feature extractor. We use the dataset as published by the original IDS-ML authors for two reasons. First, Phase 1 is a faithful reproduction of the IDS-ML baselines, which were trained and evaluated on the original CICIDS2017 release. Substituting a corrected version would break comparability with the baseline accuracies reported in the original IDS-ML paper [1]. Second, and more important for this study’s claims, the InceptionV3 baseline and the AutoML model are trained and evaluated on the identical dataset version and the identical stratified split, so any label noise penalizes both models equally and does not bias the relative comparison this paper actually makes between model size and accuracy. Repeating the full pipeline on a corrected or refactored CICIDS2017 release [43] would sharpen the absolute accuracy numbers and is identified as a concrete validation step for future work, though it is unlikely to change the relative ordering of InceptionV3 and AutoML.

Phase 3 hardware measurements used the same TFLite runtime configuration for both models (single-threaded inference, no XNNPACK delegate, identical warmup), but the full set of runtime settings is not exhaustively reported. The relative latency, CPU, and memory comparisons remain valid because both models were measured under the same conditions; absolute numbers may shift under different runtime settings.

6. Conclusions

6.1. Summary

This study successfully replicated the Transfer Learning component of the IDS-ML framework, confirming the effectiveness of deep learning CNNs adapted for intrusion detection. The InceptionV3 baseline demonstrated strong efficacy on the CICIDS2017 dataset, achieving an accuracy of 91.53%; our proposed enhancement utilizing AutoML matched this baseline’s classification performance to within sampling noise (Weighted F1 0.9203 vs. 0.9186, accuracy 91.22% vs. 91.53%; McNemar’s test $p=0.11$, see Section 4). The AutoML pipeline synthesized an optimized architecture that reduced the parameter footprint by 94.7%, from 22.3 million to 1.18 million parameters.

Edge deployment validation on a Raspberry Pi 4 confirmed that the TFLite model conversion preserves classification behavior: both models produced identical predicted classes to their Keras counterparts across all 11,316 test samples. Under sustained 41 h network traffic loads, the AutoML model achieved an average inference latency of 4.76 ms compared to InceptionV3’s 89.03 ms, an 18.7× speedup, while consuming 3.5× less CPU and 2.1× less memory. These results demonstrate that the AutoML architecture is not only comparable in classification accuracy but substantially more efficient for real-world deployment on resource-constrained hardware, supporting practical edge-based intrusion detection on resource-constrained hardware.

6.2. Future Work

While this study validated edge viability on a microprocessor-based gateway (Raspberry Pi 4), future work must extend this paradigm to the “extreme edge” by targeting heavily resource-constrained Microcontroller Units (MCUs). Specifically, future deployments should evaluate the conversion of the AutoML architecture into C++ arrays via TensorFlow Lite for Microcontrollers (TinyML) to run directly on 32-bit MCU platforms, such as the Arduino Portenta or ESP32 [44].

Industrial Operational Technology (OT) and IoT endpoints frequently operate on highly deterministic, bare-metal hardware or utilizing a Real-Time Operating System (RTOS), such as FreeRTOS or Mbed OS, possessing only kilobytes of RAM [45]. Embedding our highly compressed, 1.18 M parameter framework directly into the RTOS of an endpoint device would effectively convert vulnerable industrial sensors into self-monitoring HIDS endpoints. Future research will quantify the real-time inference latency, power consumption, and RTOS scheduler overhead of this model when subjected to physical network loads on MCU hardware [46].

Future research will also explore unsupervised anomaly detection approaches where each edge device learns normal network behavior during an initial training period and subsequently detects deviations without requiring labeled attack data, enabling detection of previously unseen zero-day threats. Combined with federated learning [47], this approach could enable convergent IT/OT threat detection, where devices across both Information Technology and Operational Technology networks collaboratively improve a shared-detection model by exchanging only model weight updates, preserving the network isolation required in IoT environments. Such capabilities could also serve as a component of Zero Trust architectures, where continuous traffic classification per device provides real-time trust assessment to inform automated access control decisions [48].

A limitation of this study is that CICIDS2017 reflects IT network traffic, not industrial OT environments. OT systems use specialized protocols (e.g., Modbus, DNP3) and face attack classes (firmware tampering, sensor spoofing) that simply do not appear in CICIDS2017. While several OT-focused datasets exist (e.g., Electra [49], X-IIoTID [50], Edge-IIoTset [51], HAI [52]), they are typically narrower in protocol coverage, smaller in scale, or specific to a single ICS process. Validating the AutoML approach for OT deployment would benefit from a comprehensive, multi-protocol OT IDS benchmark of the maturity that CICIDS2017 has provided to IT-side research, along with re-training the compressed architecture on such data.