A Deterministic Comparison of Classical Machine Learning and Hybrid Deep Representation Models for Intrusion Detection on NSL-KDD and CICIDS2017

Abstract

Intrusion detection systems (IDSs) must balance detection quality with operational transparency. We present a deterministic, leakage-free comparison of three classical classifiers: Naïve Bayes (NB), Logistic Regression (LR), and Linear Discriminant Analysis (LDA). We also propose a hybrid pipeline that trains LR on Autoencoder embeddings (AE). Experiments use NSL-KDD and CICIDS2017 under two regimes (with/without SMOTE (Synthetic Minority Oversampling Technique) applied only on training data). All preprocessing (one-hot encoding, scaling, and imputation) is fitted on the training split; fixed seeds and deterministic TensorFlow settings ensure exact reproducibility. We report a complete metric set—Accuracy, Precision, Recall, F1, Area Under the Curve (AUC), and False Alarm Rate (FAR)—and release a replication package (code, preprocessing artifacts, and saved prediction scores) to regenerate all reported tables and metrics. On NSL-KDD, AE+LR yields the highest AUC (≈0.904) and the strongest F1 among the evaluated models (e.g., 0.7583 with SMOTE), while LDA slightly edges LR on Accuracy/F1. NB attains very high Precision (≈0.98) but low Recall (≈0.24), resulting in the weakest F1, yet a low FAR due to conservative decisions. On CICIDS2017, LR delivers the best Accuracy/F1 (0.9878/0.9752 without SMOTE), with AE+LR close behind; both approach ceiling AUC (≈0.996). SMOTE provides modest gains on NSL-KDD and limited benefits on CICIDS2017. Overall, LR/LDA remain strong, interpretable baselines, while AE+LR improves separability (AUC) without sacrificing a simple, auditable decision layer for practical IDS deployment.

1. Introduction

In today’s hyperconnected society, safeguarding digital infrastructures has become a cornerstone of national, economic, and social stability. As cyber threats continue to evolve in scale and complexity, organizations must adopt intelligent and proactive defense mechanisms. Intrusion detection systems (IDSs) represent a critical layer in this defense architecture, serving to monitor and identify malicious or anomalous activity in real time. Traditionally, IDSs have relied on signature-based or rule-based techniques, which match observed patterns against known attack signatures. Although effective against previously encountered threats, these approaches often fail to detect new or obfuscated attacks and typically suffer from high false positive rates [1]. To address these limitations, recent efforts have turned to machine learning (ML), deep learning (DL), and hybrid methods capable of generalizing from data and adapting to new threat landscapes.

Machine learning models (both supervised and unsupervised) enable systems to learn behavioral patterns from historical traffic data [2]. Classical classifiers, such as Naïve Bayes (NB), Logistic Regression (LR), and Linear Discriminant Analysis (LDA), apply well-established mathematical frameworks to detect anomalies in network activity [1]. While these models are often interpretable and computationally lightweight, they can struggle with non-linear relationships and high-dimensional feature spaces typical of modern cybersecurity datasets.

In contrast, deep learning (DL) architectures provide powerful tools for automated representation learning and pattern recognition. In particular, Autoencoders (AEs) can learn compact, informative latent embeddings from network traffic data [3,4,5]. These embeddings can then be fed to simple yet effective downstream classifiers such as LR, yielding hybrid pipelines (AE+LR) that combine unsupervised feature learning with interpretable decision boundaries. Such DL-based representations have demonstrated strong performance in identifying complex and evolving cyber threats [1,6]. However, practical deployment still faces challenges related to computational demands, data requirements, and explainability [7,8].

Beyond intrusion detection, deep neural architectures have also been applied to security-critical communication scenarios, such as IRS (intelligent reflecting surfaces)-assisted NOMA (nonorthogonal multiple access) systems optimized via cooperative graph neural networks (CO-GNN) [9], which emphasize rich visualization of performance metrics to complement tabular evaluations.

In the context of applied AI and cybersecurity engineering, conducting rigorous comparative studies is vital. Such evaluations clarify when interpretable statistical models are sufficient for effective detection and when representation learning can provide measurable gains when paired with lightweight classifiers.

This study presents a deterministic comparison of three classical classifiers (NB, LR, and LDA) and a hybrid deep-learning pipeline (AE+LR) for intrusion detection [10]. To ensure robustness, we employ two widely used benchmark datasets: NSL-KDD [11] (derived from KDD’99) and CICIDS2017 [12], which include up-to-date traffic and attack types. These datasets provide a balanced basis to examine model generalization, performance across diverse attacks, and real-world feasibility.

We assess each model using standard performance metrics: Accuracy, Precision, Recall, F1-score, AUC, and FAR. In addition, we provide complete tabular reporting. The key contribution of this paper includes the following:

(i) a unified, leakage-free, and deterministic evaluation protocol applied consistently across NB, LR, LDA, and a hybrid AE+LR pipeline; (ii) dual training regimes (with/without SMOTE strictly on training folds) to isolate the effect of class rebalancing; (iii) a transparent deep-representation + linear classifier baseline (AE+LR) that combines competitive AUC/F1 with auditability; (iv) complete tables (Accuracy, Precision, Recall, F1, AUC, FAR) for both NSL-KDD and CICIDS2017; and (v) a replication package (code, configs, fixed seeds) enabling reproducibility.

From an operational standpoint, we interpret transparency as the use of models whose decision functions can be inspected, reproduced, and calibrated by security analysts. For this reason, we concentrate on linear probabilistic classifiers (LR/LDA) and on a hybrid AE+LR pipeline in which the deep Autoencoder is used solely as a representation learner, while the final decision layer remains a simple, auditable logistic regression classifier.

The remainder of this article is structured as follows. Section 2 provides a review of related research on IDS using ML and DL. Section 3 describes the experimental methodology. Section 4 presents the results. Section 5 discusses the findings, limitations, contextual relevance, recommendations and directions for future work.

2. Evolution of Machine Learning and Deep Learning for IDS

2.1. Classical ML Baselines for IDS

Early applications of machine learning to intrusion detection focused on algorithms like decision trees, support vector machines, k-nearest neighbors, Naïve Bayes, and Logistic Regression to classify network traffic as normal or attack [13]. For example, Gu and Lu [14] proposed an SVM-based IDS enhanced with Naïve Bayes feature embedding, which improved detection performance by leveraging probabilistic feature–class relationships. Traditional ML models, however, often encounter challenges with the high dimensionality and complexity of network data. Naïve Bayes (NB) assumes feature independence and can underperform when features are correlated (common in network traffic), leading to high false positives, especially on minority attack classes [1]. Logistic Regression (LR) is a linear classifier that tends to perform well only if the data is roughly linearly separable [1]. LDA (Linear Discriminant Analysis), similarly, finds linear combinations of features that separate classes; it has been used both for dimensionality reduction and classification in IDS studies. For instance, a recent work in an industrial IoT (Internet of Things) context reported ~97% detection accuracy using a wrapper-based LDA classifier on the NSL-KDD dataset [15]. Overall, while fast and interpretable, these statistical models may struggle to capture non-linear relations in network traffic features or the complex structures present in modern attack behaviors. In addition to their competitive accuracy, LR and LDA expose linear decision functions and probabilistic outputs that can be directly inspected or combined with post hoc explanation tools, which facilitates auditing and integration into existing SOC workflows.

2.2. Deep Learning Approaches and AE-Based Representation Learning

In parallel, deep learning approaches gained popularity for their ability to learn feature representations, beginning with intrusion detection using feedforward neural network (FFNN) classifiers [16,17]. Autoencoders (AEs), as unsupervised neural networks, learn to compress data into a lower-dimensional latent space and reconstruct it. They have been actively studied in IDS because they can learn to model typical network traffic and derive informative latent embeddings [18]. Several studies analyze diverse AE architectures for intrusion detection, showing that model capacity and bottleneck size strongly affect downstream performance; for example, a simple stacked AE has reported F1-scores close to 0.90 on NSL-KDD when properly tuned [18]. Other works further improve AE-based pipelines by addressing data biases and outliers, reaching accuracies around 90% on NSL-KDD [19]. While many AE studies use reconstruction-error thresholds for anomaly scoring—whose calibration can be delicate [10]—an alternative is to use AE purely as a representation learner and train a lightweight supervised classifier on top of its embeddings (e.g., AE+LR), which removes the need for threshold tuning while retaining interpretability.

2.3. Ensembles and Imbalance Remedies

Beyond single models, ensemble strategies have been explored to boost robustness. Roy et al. [20] introduced a stacking ensemble of boosted decision tree models, evaluated on IoT traffic with NSL-KDD and CICIDS2017; their system obtained a high overall accuracy of 98.5% on NSL-KDD and 99.11% on CICIDS2017, though detection of rare attack classes (user to root (U2R), remote to local (R2L)) remained challenging. De Souza et al. [21] reported a two-step ensemble (ExtraTrees and neural networks) that achieved an impressive 99.81% accuracy on NSL-KDD; however, even this advanced model detected only 68.75% of U2R attacks, underscoring the persistent difficulty of heavily imbalanced distributions. To mitigate these issues, recent work frequently incorporates data oversampling, cost-sensitive learning [22], and feature selection.

In summary, the literature indicates that learned representations from deep models—particularly Autoencoders—often enhance detection accuracy and adaptability to evolving attacks when paired with simple classifiers, while classical methods (NB, LR, LDA) remain attractive for speed, simplicity, and explainability [1]. Our study builds on these insights by directly comparing representative classical classifiers (NB, LR, LDA) against a hybrid pipeline that uses unsupervised deep representations with a linear decision layer (AE+LR) under a unified experimental setup.

3. Methodology

We design a fair, deterministic and leakage-free evaluation to compare classical machine learning baselines with a hybrid deep-representation pipeline for IDS. All preprocessing/resampling steps are fit only on training data; seeds are fixed across Python V.3.12.11/NumPy V.2.3.3/TensorFlow V.2.20.0, enabling exact reproducibility.

3.1. Evaluation Protocol and Datasets

We adopt a held-out scheme in both benchmarks: for NSL-KDD, we use the official KDDTrain+/KDDTest+ split; for CICIDS2017, we use a single 80/20 stratified split with random_state=42. We evaluate the binary setting (attack vs. normal) to focus on primary detection, consistent with prior IDS literature [1,19,20,23].

NSL-KDD (improved KDD’99) contains 41 features plus the label and four attack families [11]. Table 1 shows a summary of the information about the type of traffic contained in this dataset.

Table 1. Class distribution in the NSL-KDD dataset (KDDTrain+ and KDDTest+ combined).

Class	Samples (Train)	Samples (Test)	Total
Normal	67,343	9710	77,053
DoS	45,927	7458	53,385
Probe	11,656	2887	14,543
R2L	995	2421	3416
U2R	52	67	119
Total	125,973	22,543	148,516

Note. DoS = Denial-of-Service.

CICIDS2017 captures modern traffic/attacks with ∼80 numerical flow features [12,24]. Table 2 shows a summary of the information about the type of traffic contained in this dataset.

Table 2. Attack categories from CICIDS2017 used in this study (80/20 stratified split; natural class imbalance retained).

Category	Example Attacks	Samples (Train)	Samples (Test)
Brute Force	FTP-Patator, SSH-Patator	5578	1394
DoS/DDoS	DoS Hulk, LOIC, HOIC, DDoS	345,376	86,344
Web Attack	XSS, SQL Injection, etc.	1645	411
Botnet	ARES botnet traffic	3842	961
Infiltration	Unauthorized access (infiltration)	57,478	14,370
Heartbleed	Heartbleed exploit	9	2
Benign	Legitimate traffic	1,266,053	316,513
Total	(Benign + all attacks)	1,679,981	419,995

Note. DDoS = Distributed Denial-of-Service.

External validity caveat: NSL-KDD is legacy, and CICIDS2017 is controlled; we retain them for comparability and availability.

3.2. Preprocessing (Uniform and Leakage-Free)

Pipelines are implemented uniformly: one-hot encoding for NSL-KDD categorical fields, min–max scaling to [0, 1], and median imputation for CICIDS2017 numerics. All transformers are fit on training only and applied to the held-out test split. To mitigate imbalance, we use SMOTE on training data only (no resampling in validation/test), preserving original test distributions—a standard, transparent practice in IDS [23,25].

3.3. Models and Hyperparameters

We compare well-established baselines (NB, LR, LDA) recommended by recent surveys for transparency/reproducibility [23,26] against a hybrid deep-representation pipeline (AE+LR) supported by prior IDS evidence [26,27,28]. We deliberately use minimal, uniform tuning to avoid family-specific advantages. Table 3 shows a summary of model configurations used in this work.

Table 3. Model configurations (shared across datasets unless noted).

Model	Key Setup/Hyperparameters
Naïve Bayes (GaussianNB)	Gaussian likelihood; no tunables. Pipeline: (NSL-KDD) One-hot → Min–max → NB; (CICIDS) Imputer (median) → NB. With SMOTE: resampling applied inside the training pipeline only.
Logistic Regression (LR)	Linear classifier with L2; max_iter=1000. (NSL-KDD) One-hot → Min–max → LR. (CICIDS) Imputer(median) → LR. With SMOTE: Imputer → SMOTE → LR.
Linear Discriminant Analysis (LDA)	Shared-covariance Gaussians; regularized variant recommended for tabular IDS [29]. (NSL-KDD) default LDA on scaled one-hot. (CICIDS) solver=lsqr, shrinkage=auto after imputation; with SMOTE analogous pipeline.
Autoencoder + LR (AE+LR)	Shallow symmetric AE: Dense(64, ReLU) → Dense(32, ReLU) → Dense(64, ReLU) → Output(sigmoid). Loss: mean squared error MSE; Adam $\eta ={10}^{-3}$; batch 256; 20 epochs; validation_split=0.1; shuffle=False. Train AE on training inputs only, extract $z=f_{\mathrm{AE}}\left(x\right)$; fit LR(max_iter=1000) on $z_{\mathrm{train}}$. SMOTE (when used) is applied to the embeddings $z_{\mathrm{train}}$.
SMOTE (train only)	random_state=42, k_neighbors=5, sampling_strategy=’auto’. No resampling on test.

3.4. Metrics and Reproducibility

We report Accuracy, Precision, Recall, F1, AUC (from predict_proba) and False Alarm Rate (FAR), with

$$\mathrm{FAR}={\displaystyle \frac{FP}{FP+TN}}$$

(1)

where $FP$ represents the number of false positives and $TN$ the number of true negatives.

Reporting Recall together with FAR for all models (Table 4, Table 5, Table 6 and Table 7) directly quantifies the trade-off between missed attacks and alert volume. For instance, NB attains very low FAR at the cost of extremely low Recall on NSL-KDD, whereas LR, LDA, and AE+LR achieve substantially higher Recall with modest FAR, offering more favorable operating points for real deployments.

Deterministic TensorFlow ops are enabled; exact seeds, scripts, and stored prediction scores are released in a replication package (code/configs/predict_proba), enabling byte-for-byte regeneration of tables/metrics.

All code, preprocessing artifacts, trained models, and configuration files are publicly available in our replication repository at https://github.com/miguelarcosa/IDS-KDD-CICIDS2017.git (accessed on 1 November 2025) [30], ensuring full byte-level reproducibility of all tables and metrics reported in this paper.

In addition to scalar metrics, we visualize model behavior on the held-out test splits by plotting receiver operating characteristic (ROC) curves for the four pipelines (NB, LR, LDA, AE+LR). ROC curves are computed from the saved predict_proba scores in our replication package using the scikit-learn implementation, and are shown for the regime without SMOTE, where the main AUC differences appear. The full set of confusion matrices is shown in Appendix A.

4. Results

This section reports the performance of the four evaluated approaches: Naïve Bayes (NB), Logistic Regression (LR), Linear Discriminant Analysis (LDA), and the hybrid Autoencoder embeddings + Logistic Regression (AE+LR) in NSL-KDD and CICIDS2017 datasets. We present Accuracy, Precision, Recall, F1, AUC, and False Alarm Rate (FAR), consistent with the evaluation protocol described earlier. Unless noted, models use default thresholds (0.5 for probabilistic outputs), and all preprocessing/SMOTE steps are fit on the training split only.

4.1. Overall Performance on NSL-KDD

Table 4 and Table 5 summarize the binary results on NSL-KDD without and with SMOTE, respectively. Across both regimes, LR, LDA, and AE+LR clearly outperform NB, which shows very high Precision but poor Recall (i.e., conservative attack labeling that misses many positives).

For reference, after applying SMOTE only on the training split, the NSL-KDD training set becomes exactly balanced at Benign:Attack = 80,892:80,892 (1:1).

Table 4. Performance metrics of all models on the NSL-KDD test set (without SMOTE).

Model	Accuracy	Precision	Recall	F1-Score	FAR	AUC
Naïve Bayes	0.5649	0.9800	0.2406	0.3864	0.0065	0.7987
Logistic Regression (LR)	0.7443	0.9142	0.6079	0.7302	0.0754	0.8276
LDA	0.7616	0.9249	0.6326	0.7514	0.0679	0.8484
Autoencoder (AE) + LR	0.7616	0.9165	0.6397	0.7534	0.0770	0.9040

Table 4. Performance metrics of all models on the NSL-KDD test set (without SMOTE).

Model	Accuracy	Precision	Recall	F1-Score	FAR	AUC
Naïve Bayes	0.5649	0.9800	0.2406	0.3864	0.0065	0.7987
Logistic Regression (LR)	0.7443	0.9142	0.6079	0.7302	0.0754	0.8276
LDA	0.7616	0.9249	0.6326	0.7514	0.0679	0.8484
Autoencoder (AE) + LR	0.7616	0.9165	0.6397	0.7534	0.0770	0.9040

Table 5. Performance metrics of all models on the NSL-KDD test set (with SMOTE on training data only).

Model	Accuracy	Precision	Recall	F1-Score	FAR	AUC
Naïve Bayes	0.5650	0.9800	0.2408	0.3866	0.0065	0.7978
Logistic Regression (LR)	0.7458	0.9137	0.6112	0.7324	0.0763	0.8227
LDA	0.7632	0.9238	0.6365	0.7537	0.0694	0.8525
Autoencoder (AE) + LR	0.7654	0.9166	0.6467	0.7583	0.0778	0.9043

Table 5. Performance metrics of all models on the NSL-KDD test set (with SMOTE on training data only).

Model	Accuracy	Precision	Recall	F1-Score	FAR	AUC
Naïve Bayes	0.5650	0.9800	0.2408	0.3866	0.0065	0.7978
Logistic Regression (LR)	0.7458	0.9137	0.6112	0.7324	0.0763	0.8227
LDA	0.7632	0.9238	0.6365	0.7537	0.0694	0.8525
Autoencoder (AE) + LR	0.7654	0.9166	0.6467	0.7583	0.0778	0.9043

Key observations about the results shown in Table 4 and Table 5:

• NB: Exceptionally high Precision (≈0.98) but low Recall (≈0.24) produces the lowest F1; FAR is minimal due to the conservative decision boundary.
• LR vs. LDA: LDA edges LR slightly on Accuracy/F1 and AUC in both regimes; differences are modest.
• AE+LR: Delivers the highest AUC (≈0.904) and the best Recall/F1 trade-off among the four, with Accuracy comparable to LDA.
• Effect of SMOTE: Improvements are small but consistent where they appear (e.g., LDA F1: 0.7514→0.7537; AE+LR F1: 0.7534→0.7583), indicating mild gains in detecting attacks at an approximately unchanged FAR.

On NSL-KDD, Figure 1 complements Table 4 and Table 5 by showing that the AE+LR pipeline not only attains the highest AUC (approximately 0.904) but also yields a ROC curve that lies above those of LR and LDA for most operating points, while NB lags behind, particularly in the mid-to-high FPR region. This confirms that AE+LR achieves the best overall separability between normal and attack traffic among the evaluated models.

Figure 1. ROC curves on the NSL-KDD test split (without SMOTE) for Naïve Bayes (NB), Logistic Regression (LR), Linear Discriminant Analysis (LDA), and the hybrid Autoencoder + LR (AE+LR). The AE+LR curve attains the highest AUC (approximately 0.904) and lies above those of LR and LDA for most operating points, illustrating its improved separability compared to the purely linear baselines.

4.2. Overall Performance on CICIDS2017

Table 6 and Table 7 summarize binary results on CICIDS2017. Performances are generally high across LR, LDA, and AE+LR, with NB trailing due to a high FAR and lower Precision.

Table 6. Performance metrics of all models on the CICIDS2017 dataset (without SMOTE).

Model	Accuracy	Precision	Recall	F1-Score	FAR	AUC
Naïve Bayes	0.8565	0.6364	0.9742	0.7699	0.1820	0.9711
Logistic Regression (LR)	0.9878	0.9722	0.9783	0.9752	0.0091	0.9961
LDA	0.9784	0.9426	0.9715	0.9569	0.0193	0.9923
Autoencoder (AE) + LR	0.9859	0.9691	0.9738	0.9715	0.0101	0.9960

Table 6. Performance metrics of all models on the CICIDS2017 dataset (without SMOTE).

Model	Accuracy	Precision	Recall	F1-Score	FAR	AUC
Naïve Bayes	0.8565	0.6364	0.9742	0.7699	0.1820	0.9711
Logistic Regression (LR)	0.9878	0.9722	0.9783	0.9752	0.0091	0.9961
LDA	0.9784	0.9426	0.9715	0.9569	0.0193	0.9923
Autoencoder (AE) + LR	0.9859	0.9691	0.9738	0.9715	0.0101	0.9960

Table 7. Performance metrics of all models on the CICIDS2017 dataset (with SMOTE on training data only).

Model	Accuracy	Precision	Recall	F1-Score	FAR	AUC
Naïve Bayes	0.8564	0.6362	0.9742	0.7697	0.1821	0.9686
Logistic Regression (LR)	0.9862	0.9623	0.9824	0.9723	0.0126	0.9962
LDA	0.9735	0.9156	0.9830	0.9481	0.0296	0.9924
Autoencoder (AE) + LR	0.9806	0.9428	0.9805	0.9613	0.0194	0.9954

Table 7. Performance metrics of all models on the CICIDS2017 dataset (with SMOTE on training data only).

Model	Accuracy	Precision	Recall	F1-Score	FAR	AUC
Naïve Bayes	0.8564	0.6362	0.9742	0.7697	0.1821	0.9686
Logistic Regression (LR)	0.9862	0.9623	0.9824	0.9723	0.0126	0.9962
LDA	0.9735	0.9156	0.9830	0.9481	0.0296	0.9924
Autoencoder (AE) + LR	0.9806	0.9428	0.9805	0.9613	0.0194	0.9954

Key observations about the results shown in Table 6 and Table 7:

• LR and AE+LR: Both achieve a near-ceiling AUC (≈0.996). LR attains the best overall Accuracy/F1 (0.9878/0.9752) without SMOTE; AE+LR is a close second (0.9859/0.9715).
• LDA: Strong but consistently below LR/AE+LR in this dataset, with slightly higher FAR.
• NB: High Recall (≈0.97) but much lower Precision and a high FAR (≈0.18), yielding the lowest F1.
• Effect of SMOTE: On CICIDS2017 (already balanced at the aggregate level after unification), SMOTE slightly reduces Accuracy/F1 for LR and AE+LR and increases FAR, which is consistent with minor overcompensation when the base split is relatively well balanced.

On CICIDS2017, the impact of SMOTE is inherently constrained by the near-ceiling performance already achieved by LR, LDA, and AE+LR. As reported in Table 6 and Table 7, these models reach ROC AUC values close to 0.996 and very high F1-scores even without oversampling, and SMOTE only produces small fluctuations at the third or fourth decimal place. In this regime, the decision boundary already separates benign and attack traffic very effectively, so synthetic minority examples generated by SMOTE tend to lie in regions of the feature space that are already densely populated by correctly classified points, adding little new information. In addition, some attack subcategories appear with very few and potentially noisy samples, making them poor candidates for interpolation: oversampling around isolated, overlapping, or mislabeled points may fail to improve, or even slightly perturb, the classifier’s decision surface. These observations clarify that, for CICIDS2017, SMOTE operates under conditions where its theoretical benefits are limited, and its role is mainly to provide a consistent comparison with NSL-KDD rather than to drive substantial performance gains.

For CICIDS2017, Figure 2 confirms that LR, LDA, and AE+LR all operate in a near-ceiling regime, with ROC curves that are almost indistinguishable (AUC $\approx 0.996$ for LR and AE+LR; Table 6 and Table 7). NB, in contrast, traces a noticeably lower ROC curve, reflecting its higher FAR and reduced Precision despite very high Recall. Overall, once the feature space is well separated, the additional non-linear representation from AE does not translate into a visible ROC advantage over LR on this benchmark, even though AE+LR still matches LR in AUC.

Figure 2. ROC curves on the CICIDS2017 test split (without SMOTE) for Naïve Bayes (NB), Logistic Regression (LR), Linear Discriminant Analysis (LDA), and the hybrid Autoencoder + LR (AE+LR). The AE+LR curve attains the highest AUC and lies above those of LR and LDA for most operating points, illustrating its improved separability compared to the purely linear baselines.

Runtime Analysis

Table 8 shows that, on the smaller NSL-KDD benchmark, all models train in less than 26 s and complete inference on the entire test split in at most 0.084 s. On the larger CICIDS2017 dataset, training times increase by roughly two orders of magnitude due to the 1.68 million training flows, yet even the most expensive configuration (AE+LR with SMOTE) finishes training in about 421 s (approximately 7 min). Inference on CICIDS2017 remains inexpensive for the classical models (no more than 1.9 s to score the whole test split), and although AE+LR requires about 71–73 s to compute embeddings and predictions for the 419,995 test flows, this still corresponds to sub-millisecond latency per flow. Overall, all four pipelines keep the runtime at a level compatible with near real-time IDS deployment on commodity hardware. These results are shown in Appendix B, in Table A17, Table A18 and Table A19.

Table 8. Training and inference time per model and dataset (median over three runs, in seconds).

	NSL-KDD (s)		CICIDS2017 (s)
Model	Train	Inference	Train	Inference
Without SMOTE
Naïve Bayes	0.195	0.084	39.343	0.567
Logistic Regression (LR)	1.676	0.010	98.018	1.586
Linear Discriminant Analysis (LDA)	2.644	0.007	19.742	0.559
AE+LR	18.349	0.006	217.472	71.092
With SMOTE
Naïve Bayes	5.509	0.079	386.784	1.913
Logistic Regression (LR)	10.016	0.006	391.529	0.567
Linear Discriminant Analysis (LDA)	8.177	0.010	374.988	0.756
AE+LR	25.702	0.010	421.448	73.305

4.3. Takeaways

Across both datasets, NB provides a fast but conservative baseline with high Precision and low Recall. LDA is a strong classical competitor and slightly edges LR on NSL-KDD, whereas LR leads on CICIDS2017. The hybrid AE+LR consistently offers the best AUC and competitive F1, suggesting that unsupervised embeddings help linear classifiers capture non-linear structure without sacrificing interpretability. By using the autoencoder purely to learn latent embeddings and delegating all final decisions to a linear LR classifier, AE+LR improves separability (AUC/F1) over LR/LDA while preserving the same transparent, feature-weight–based decision layer that operators can inspect, calibrate, and document. SMOTE yields modest gains in NSL-KDD (where minority attacks are scarce) and does not help—and can slightly hurt—on the unified CICIDS2017 split. These patterns align with prior IDS evidence: classical linear/probabilistic baselines remain valuable, and hybrid deep representations can further enhance linear decision functions under tabular traffic features.

4.3.1. Comparison with Related Work

It is instructive to situate our findings alongside recent IDS evaluations on NSL-KDD and CICIDS2017. Broadly, prior works report that deep learning (DL) models frequently attain near-ceiling discrimination on these corpora, whereas well-regularized linear/probabilistic baselines remain competitive, transparent references. A compact side-by-side comparison with recent IDS studies is summarized in Table 9.

Table 9. Compact comparison with related IDS studies on NSL-KDD/CICIDS2017 (binary settings when reported).

Study	Dataset	Method	Headline Metric (Decimal, 4 d.p.)
Ali et al. [1]	Mixed	LR/DL (deep learning models such as multilayer perceptrons (MLP) and convolutional neural networks (CNN))	LR Acc ≈ 0.9700; DL Acc ≈ 0.9800
Umer & Abbasi [31]	NSL-KDD	AE+LSTM	Acc ≈ 0.8900 (low FAR priority)
Chen et al. [32]	CICIDS2017	DBN+LSTM	Acc = 0.8680
Roy et al. [20]	CICIDS2017	Lightweight ensemble	Acc = 0.9911
This work	NSL-KDD	AE+LR	AUC = 0.9043; F1 = 0.7583 (with SMOTE; train-only)
This work	CICIDS2017	LR/AE+LR	Acc = 0.9878/0.9859; AUC = 0.9961/0.9960 (no SMOTE; train-only transforms)

Linear/Probabilistic baselines. Ali et al. [1] report LR around ∼97% Accuracy, and NB near ∼64% on their benchmark—trends that mirror our binary CICIDS2017 results, where LR achieves 98.78% Accuracy (F1 = 0.9752), and NB substantially trails due to its low Precision and high FAR (Table 6). Our LDA is also competitive, consistent with surveys that position LR/LDA as strong, reproducible tabular baselines [23,26].

Deep learning and hybrids. Several studies show DL advantages on NSL-KDD/CICIDS2017. Umer and Abbasi [31] report an AE–LSTM (Long Short-Term Memory) two-stage system with ∼89% accuracy on NSL-KDD, prioritizing low false alarms; Chen et al. [32] obtain 86.8% on CICIDS2017 with a Deep Belief Network (DBN)+LSTM; and Roy et al. [20] reach 99.11% on CICIDS2017 using a lightweight supervised ensemble. Rather than training a full end-to-end DL classifier, our study adopts a hybrid approach (AE+LR): an autoencoder provides unsupervised embeddings and a linear LR supplies an auditable decision layer. This yields near-ceiling AUC on CICIDS2017 (AUC ≈ 0.996) and the top AUC on NSL-KDD among our four families (AUC ≈ 0.904), aligning with evidence that AE-driven representation learning can improve linear separability while preserving interpretability [5,19,23,26,27].

On metric deltas across papers. Absolute numbers in cross-paper comparisons routinely vary with dataset curation (binary vs. multi-class), train/test protocol (official splits vs. random), feature engineering and normalization, imbalance remedies, and leakage controls. For instance, our pipelines fit all preprocessing (and SMOTE when used) strictly on training data and evaluate on held-out splits, which can produce more conservative but comparable scores. Prior work also documents that SMOTE often helps minority Recall on NSL-KDD but has mixed value on CICIDS2017, depending on the split [23,25]. Within this context, our results corroborate the prevailing ranking—NB ≪ LR/LDA ≲ DL—while showing that a simple AE+LR hybrid can deliver DL-like AUC with a transparent classifier.

4.3.2. Key Takeaways

In summary, no single model is universally best for all deployment constraints. Among classical baselines, LR and LDA provide strong, transparent performance with low computational cost, as confirmed by the training and inference times in Table 8, making them attractive when interpretability and latency are priorities [23]. NB is generally not competitive on modern IDS corpora unless paired with additional design choices (e.g., feature engineering or hybridization; see Gu & Lu [14]).

Our hybrid pipeline (AE+LR) shows that using an autoencoder purely as an unsupervised representation learner can deliver near state-of-the-art AUC on CICIDS2017 while preserving an auditable linear decision layer, aligning with reports that deep representations can improve linear separability without sacrificing explainability [23,26].

Operationally, the results underscore two practical points: (i) addressing class imbalance (e.g., with SMOTE applied only on training data) improves minority-class Recall on NSL-KDD, whereas its benefit on CICIDS2017 depends on the split; and (ii) reporting FAR alongside Recall is essential to balance missed attacks against analyst workload. Overall, a layered IDS that uses fast statistical models for coarse filtering and learned representations for refined decisions can leverage complementary strengths under realistic constraints.

4.4. Statistical Robustness and Uncertainty Quantification

4.4.1. Statistical Analysis on NSL-KDD

The statistical evaluation on the NSL-KDD dataset, presented in Table A18, reveals varying levels of stability across the tested models:

• High Stability in LR and NB: The Logistic Regression (LR) and Naïve Bayes (NB) models demonstrate extreme stability, particularly when run without SMOTE. Their SD values are negligible (SD < 0.0001 for most metrics), and the 95% CIs are extremely narrow (e.g., CI_Acc. = [0.7443, 0.7443] for LR without SMOTE). This confirms that the deterministic results initially reported for these linear models are highly reliable.
• Hybrid Model Consistency: The Autoencoder-based hybrid models (LR + AE) also exhibit high consistency, showing low SD values (SD_Acc. ≈ 0.0008) and tight 95% CIs. This indicates that the observed performance improvement from deep representation learning is statistically robust and consistent across different initialization states.
• Highest Instability in LDA: In contrast, the Linear Discriminant Analysis (LDA) model without SMOTE displays the highest sensitivity to the random seed. The SD_Acc. is significantly higher (0.1390), resulting in a wide Range_Acc. of [0.5650, 0.7616]. This high variability suggests that while LDA can occasionally achieve high performance, its average performance and practical reliability on this dataset are inconsistent.

Overall, the statistical analysis confirms that the primary conclusions regarding the relative performance and stability of the models on NSL-KDD, especially the consistent performance of the LR + AE hybrid approach, are statistically sound and independent of the random seed, except for the volatile LDA model.

4.4.2. Statistical Analysis on CICIDS2017

The statistical assessment on the CICIDS2017 dataset, presented in Table A19, demonstrates a remarkably high degree of stability across all tested models:

• Near-Zero Variance Across All Models: Both classical and hybrid models exhibit exceptional consistency, with SD values generally below 0.0005 for key metrics like Accuracy, Precision, and F1-score. For instance, LR without SMOTE shows SD ≤ 0.0004 across all reported metrics. Even the deep learning-based LR + AE models, which typically have greater stochasticity, maintain very low SDs (SD_Acc. ≤ 0.0005).
• Statistical Significance of Performance Differences: The clear and high separation in the mean performance between the top-performing models (e.g., LR without SMOTE with Mean_Acc. = 0.9877) and the lower-performing classical models (e.g., LDA without SMOTE with Mean_Acc. = 0.8529) is validated by the non-overlapping 95% CIs. This confirms that the ranking of models is statistically significant.
• Decisive Robustness: The very low uncertainty across the board confirms that the high detection quality (e.g., AUC ≥ 0.99 for top models) is an intrinsic and robust property of the models on the CICIDS2017 dataset.

These results are shown in Appendix B.

5. Conclusions and Future Work

This study presented a rigorous, reproducible comparison of three classical clas- sifiers—Naïve Bayes (NB), Logistic Regression (LR), and Linear Discriminant Analysis (LDA)—and a hybrid deep-representation pipeline based on Autoencoder embeddings with a linear classifier (AE+LR). Using the NSL-KDD and CICIDS2017 benchmarks, we reported Accuracy, Precision, Recall, F1, AUC, and False Alarm Rate (FAR), with all preprocessing and any class rebalancing (SMOTE) fit strictly on the training split. In addition, we measured wall-clock training and inference times for all four pipelines (Table 8), showing that NB, LR, and LDA train in seconds on NSL-KDD and in at most a few hundred seconds on CICIDS2017, while inference for all models, including AE+LR, remains well below one millisecond per flow, which is compatible with near real-time IDS operation on commodity hardware.

• What performed best and where.

On NSL-KDD, AE+LR and LDA offered the strongest overall trade-offs. AE+LR attained the highest AUC (≈0.904; Table 5) and the best F1 among the four models in both regimes (e.g., 0.7583 with SMOTE), while LDA slightly edged LR on Accuracy/F1 and AUC (e.g., F1 0.7537 vs. 0.7324 with SMOTE). NB achieved very high Precision (≈0.98) but very low Recall (≈0.24), yielding the weakest F1; its low FAR reflects a conservative decision boundary that misses many attacks.

On CICIDS2017, performance was strong for LR, LDA, and AE+LR. LR achieved the best Accuracy/F1 without SMOTE (0.9878/0.9752), with AE+LR a close second (0.9859/0.9715), and both near-ceiling AUC (≈0.996). NB again trailed due to a much higher FAR (≈0.18) and lower Precision, despite high Recall.

• The effect of class rebalancing.

Applying SMOTE to training data produced modest gains on NSL-KDD (where rare attacks are genuinely scarce), e.g., small F1 improvements for LDA and AE+LR with essentially unchanged FAR. In the unified binary split of CICIDS2017, SMOTE did not help and in some cases slightly reduced Accuracy/F1 while increasing FAR—consistent with mild overcompensation when the base class balance is already adequate.

• Practical implications.

For operators seeking transparent and lightweight deployment, LDA and LR remain strong, reproducible baselines. Where a small computational overhead is acceptable, AE+LR provides additional separability (highest AUC) and competitive F1 by leveraging an unsupervised representation while keeping a simple, auditable linear classifier. NB remains useful as a sanity-check baseline and for resource-constrained scenarios, but its low Recall on NSL-KDD cautions against relying on it as a stand-alone detector.

• Limitations.

Our primary focus was the binary setting; then, a complete multiclass analysis and cost-sensitive operating points were not the central objective here. In addition, AE was used strictly as a feature learner (not as a thresholded anomaly detector), so conclusions about reconstruction-threshold tuning are beyond scope.

Future Work

• Multiclass and cost-aware evaluation. Extend to full multiclass NSL-KDD with macro/micro metrics and per-class PR (Precision–Recall) curves; explore cost-sensitive thresholds (e.g., penalizing FNs (false negatives) for critical attacks).
• Non-linear classical baselines. Include Support Vector Machines (with kernels) and tree ensembles (Random Forest, Gradient Boosting) as non-linear yet interpretable tabular baselines.
• Representation learning variants. Compare AE with denoising/variational variants and different bottleneck sizes; quantify when AE+LR’s gains over LR/LDA justify the added cost.
• Imbalance strategies. Evaluate cost-sensitive learning, probability calibration, and focal-type objectives; compare SMOTE with alternatives such as Synthetic Minority Over-sampling Technique for Nominal and Continuous features (SMOTE-NC) and Adaptive Synthetic Sampling (ADASYN) and informed under-sampling.
• Cross-dataset generalization. Train on one domain and test on another (e.g., NSL-KDD → CICIDS2017 or the UNSW-NB15 intrusion detection dataset) with domain adaptation/transfer learning; examine robustness to concept drift.
• Operationalization and explainability. Integrate local explanations, such as Shapley Additive exPlanations (SHAP) for LR/LDA and in the AE latent space, into alerting workflows; these additions extend our focus on operational transparency by enabling analysts to see which features drive individual alerts and to validate or refine decision policies accordingly.
• Reproducible pipelines. Release notebooks that regenerate all tables and reported metrics from predict_proba/decision_function; add regression tests to guard against library drift.
• Cross-dataset validation. Replicate the full pipeline on UNSW-NB15 and the CIC-IDS-2018 dataset from the Canadian Institute for Cybersecurity (CIC) and run train→test cross-domain transfers to assess robustness under dataset shift.
• Dataset fusion and harmonization. Explore concatenating harmonized feature-spaces from NSL-KDD and CICIDS2017 (with a domain indicator) to train unified models, and quantify trade-offs in Accuracy/FAR vs. domain shift. Evaluate whether AE-based representations ease cross-domain fusion.

• Overall, the results support a pragmatic recommendation: start with LR/LDA for simplicity and transparency, and adopt AE+LR when consistent separability gains (AUC) are desired without abandoning a linear, interpretable classifier. Careful use of imbalance remedies and alarm-centric metrics (Recall and FAR) is essential for reliable IDS deployment.