Review Reports - A Comprehensive Review: The Evolving Cat-and-Mouse Game in Network Intrusion Detection Systems Leveraging Machine Learning

Round 1

Reviewer 1 Report

Sections 4 (Adversarial Attacks) and 5 (Defenses) predominantly list studies without synthesizing trends, efficacy, or limitations. For instance: Table 4 catalogs attack techniques but omits comparative analysis of their real-world feasibility (e.g., computational cost, detectability). Section 5.4 briefly mentions "unified defenses" but does not evaluate why hybrid approaches remain underexplored (Discussion, §6).

Key terms are inconsistently applied. Examples: "Gray-box attack" is defined in §2.2.1 as requiring partial knowledge (e.g., training data access) but used interchangeably with "limited knowledge" in §4.3. The description of ZOO (Zeroth-Order Optimization) in §2.3.2 misrepresents its query efficiency. The paper states it requires "significant computational time" (p. 8), but recent studies optimize this via Hessian approximation—unaddressed here.

Table 3 lists dataset pros/cons but fails to contextualize how obsolete data skew adversarial robustness claims. The Discussion (§6) notes this issue but does not tie it to experimental validity in cited studies.

Section 5 catalogs defenses but omits quantitative comparisons. For instance, DeepFool-based defenses (§5.1.2) claim superiority over GANs but lack metrics (e.g., exact detection-rate improvements from [62]). Table 5 lists defenses but omits success rates against combined (evasion+poisoning) attacks—a gap highlighted in §6.

The Discussion (§6) identifies gaps (e.g., physical-layer attacks, unified defenses) but lacks specificity. For example, the recommendation to "develop unified defenses" does not reference emerging frameworks (e.g., ensemble-based [150]). "Real-time adversarial example generation" is proposed without technical pathways (e.g., hardware acceleration).

Author Response

Reviewer 1:

Sections 4 (Adversarial Attacks) and 5 (Defenses) predominantly list studies without synthesizing trends, efficacy, or limitations. For instance: Table 4 catalogs attack techniques but omits comparative analysis of their real-world feasibility (e.g., computational cost, detectability). Section 5.4 briefly mentions "unified defenses" but does not evaluate why hybrid approaches remain underexplored (Discussion, §6).

Thank you very much for your valuable comments. We have revised sections 4 and 5 to include synthesis of trends, e.g., increasing prevalence of the GAN-based attacks in the black-box attacks, efficacy, e.g., the evasion attack rates in IoT NIDS, and limitations, e.g., high query costs in the ZOO attacks. Table 4 now includes new columns for "Computational Cost" and "Detectability". Also, the subsection 5.4 and the discussion section now explain why the hybrid approaches are underexplored, e.g., due to the high complexity in integrating evasion and poisoning defense, including the emerging frameworks.

Key terms are inconsistently applied. Examples: "Gray-box attack" is defined in §2.2.1 as requiring partial knowledge (e.g., training data access) but used interchangeably with "limited knowledge" in §4.3. The description of ZOO (Zeroth-Order Optimization) in §2.3.2 misrepresents its query efficiency. The paper states it requires "significant computational time" (p. 8), but recent studies optimize this via Hessian approximation—unaddressed here.

We have fixed all of these issues and highlighted the revised parts in the manuscript.

Table 3 lists dataset pros/cons but fails to contextualize how obsolete data skew adversarial robustness claims. The Discussion (§6) notes this issue but does not tie it to experimental validity in cited studies.

We have updated Table 3 and the discussion section and highlighted the changes to tie it to the experimental validation.

Section 5 catalogs defenses but omits quantitative comparisons. For instance, DeepFool-based defenses (§5.1.2) claim superiority over GANs but lack metrics (e.g., exact detection-rate improvements from [62]). Table 5 lists defenses but omits success rates against combined (evasion+poisoning) attacks—a gap highlighted in §6.

Thank you for your noticeable comments. We have revised section 5 to include the quantitative comparisons and updated Table 5, which now has a new column "Success Rate vs. Combined Attacks", e.g., for the hybrid ensembles: "20-30% improvement against the evasion+poisoning [150]". We further highlighted this gap and mentioned the hybrid improvements in the discussion section.

5. The Discussion (§6) identifies gaps (e.g., physical-layer attacks, unified defenses) but lacks specificity. For example, the recommendation to "develop unified defenses" does not reference emerging frameworks (e.g., ensemble-based [150]). "Real-time adversarial example generation" is proposed without technical pathways (e.g., hardware acceleration).

(1) Addressing the Lack of the Specificity on "Unified Defenses" We have added to the manuscript the following “Unified defenses against both the evasion and poisoning attacks, consistent with the 2025 NIST taxonomy [10], can be implemented leveraging the emerging ensemble-based frameworks [131]. For instance, the hybrid adversarial-training ensembles have been shown to achieve 20–30% improvements in the robustness of the model.” (2) Addressing the Lack of the Specificity on "Real-Time Adversarial Example Generation" We have added to the manuscript the following “These defensive approaches can be further strengthened through incorporating the technical pathways, including the hardware acceleration, e.g., FPGA and GPU pipelines for the real-time adversarial example generation [60]. For the real-time generation, the pathways should include GPU or FPGA acceleration since they can be used to reduce the latency by approximately 50% in the SDN testbeds [104, 60].” (3) Additional Added Related Improvements to the manuscript We have added to the manuscript the following “Bridge academic-real-world gaps with physical-layer attacks, informed by 2025 reviews on NIDS-specific adversarial impacts [14]. For example, the physical-layer simulations, including the packet-level perturbation models that are deployed in the SDN testbeds [103], enable the real evaluation of the adversarial traffic behavior in the operational network conditions.”

Reviewer 2 Report

The paper provides a comprehensive overview of adversarial attacks and defenses in ML-based NIDS. However, the organization could be improved by adding subheadings under major sections (e.g., attack types, defense strategies) to enhance readability and flow.

The paper provides a comprehensive overview of adversarial attacks and defenses in ML-based NIDS. However, the organization could be improved by adding subheadings under major sections (e.g., attack types, defense strategies) to enhance readability and flow.
The motivation is clearly stated, and the contributions are well-defined. However, the paper would benefit from a clearer statement of how it advances beyond existing surveys, especially given the references to several prior reviews.
The review covers a wide range of attack methods (e.g., GANs, ZOO, DeepFool, FGSM) and defenses, with useful mathematical formulations. However, some sections (e.g., Section 3 on datasets) are overly descriptive and could be condensed.
Tables 2, 3, 4, and 5 are valuable for summarizing techniques, datasets, and studies. However, the analysis would be stronger with more critical insights into why certain methods perform better or worse in specific contexts.
The inclusion of 2024 and 2025 references (e.g., [10], [14], [150]) is a strength, but some older datasets (e.g., KDDCup99) are still heavily cited despite being outdated. The paper should better justify their use or emphasize modern alternatives.
The discussion identifies important gaps (e.g., lack of unified defenses, reliance on outdated datasets) and suggests future directions. However, it could be more structured—perhaps as a bulleted list of research challenges.
The paper contains several grammatical errors and awkward phrasings (e.g., "metile adversarial attacks", "real functional behavior of network traffic"). A thorough proofreading is recommended.
Figures 1 and 2 are referenced but not included in the provided text. Their absence limits the ability to assess their relevance and clarity. Ensure all figures are included and properly explained.
The paper gives substantial attention to attack methods but could expand more on defensive strategies, especially hybrid or unified defense frameworks that address both poisoning and evasion.
The conclusion is concise but could better summarize the key findings and their implications for future research and practical deployment of robust ML-based NIDS.

Author Response

Reviewer 2:

The paper provides a comprehensive overview of adversarial attacks and defenses in ML-based NIDS. However, the organization could be improved by adding subheadings under major sections (e.g., attack types, defense strategies) to enhance readability and flow.

Thank you very much for your great comments.

Details:

The paper provides a comprehensive overview of adversarial attacks and defenses in ML-based NIDS. However, the organization could be improved by adding subheadings under major sections (e.g., attack types, defense strategies) to enhance readability and flow.

We have addressed this comment, updated, and highlighted the changes in the manuscript accordingly.

The motivation is clearly stated, and the contributions are well-defined. However, the paper would benefit from a clearer statement of how it advances beyond existing surveys, especially given the references to several prior reviews.

We have revised the motivation part to show the main differences between this work and other related works.

The review covers a wide range of attack methods (e.g., GANs, ZOO, DeepFool, FGSM) and defenses, with useful mathematical formulations. However, some sections (e.g., Section 3 on datasets) are overly descriptive and could be condensed.

Thank you for your comments. We reduced and updated section 3. More specifically, Section 3 has been condensed by summarizing pros/cons in Table 3 and combining descriptive paragraphs into accurate overviews.

Tables 2, 3, 4, and 5 are valuable for summarizing techniques, datasets, and studies. However, the analysis would be stronger with more critical insights into why certain methods perform better or worse in specific contexts.

These Tables have been revised to include critical insights by adding new columns and analysis to show why such methods perform better or worse than others in the specific contexts.

The inclusion of 2024 and 2025 references (e.g., [10], [14], [150]) is a strength, but some older datasets (e.g., KDDCup99) are still heavily cited despite being outdated. The paper should better justify their use or emphasize modern alternatives.

The revised section 3 and the discussion now justify the older datasets for baseline comparison and confirm leveraging modern ones.

The discussion identifies important gaps (e.g., lack of unified defenses, reliance on outdated datasets) and suggests future directions. However, it could be more structured—perhaps as a bulleted list of research challenges.

We have explained this and revised the discussion accordingly.

The paper contains several grammatical errors and awkward phrasings (e.g., "metile adversarial attacks", "real functional behavior of network traffic"). A thorough proofreading is recommended.

We fixed the grammatical mistakes and the typos and carefully proofread the entire manuscript.

Figures 1 and 2 are referenced but not included in the provided text. Their absence limits the ability to assess their relevance and clarity. Ensure all figures are included and properly explained.

All Figures, including Figures 1 and 2, are now included with clear captions and explanations. We also clearly included them in the provided text.

The paper gives substantial attention to attack methods but could expand more on defensive strategies, especially hybrid or unified defense frameworks that address both poisoning and evasion.

We have expanded the defensive technique section further and included more details about the new and hybrid defenses in section 5 and the discussion section.

The conclusion is concise but could better summarize the key findings and their implications for future research and practical deployment of robust ML-based NIDS.

The conclusion has been revised to include the key findings, and further future works have been added.

Reviewer 3 Report

This paper aimed to integrate machine learning into the network intrusion detection system to build more accurate and adaptive defense mechanisms. Given the current high performance and robustness of machine learning-based network intrusion detection systems, this manuscript emphasized the development of more robust technologies to withstand these attacks. Authors should respond to the following four comments:

The authors should discuss the most common attack techniques and the most common defensive measures for network intrusion detection systems.
The formulas should be revised and reorganized for readers.
The conclusions should emphasize supervised learning, unsupervised learning, and deep learning in machine learning (ML) for network intrusion detection systems (NIDS).
The authors should discuss the future of the intrusion detection system and the future prospects for the cat and mouse game.

Since this is a review paper, I hope the authors will correct the English and formulas themselves. Additionally, authors should revise the introduction and conclusion for the benefit of the readers.

Comments for author File: Comments.pdf

Author Response

Reviewer 3:

The authors should discuss the most common attack techniques and the most common defensive measures for network intrusion detection systems.

Thank you for your comments. We have revised the manuscript and highlighted the most common attack and defense techniques in NIDS in sections 4 and 5, respectively.

The formulas should be revised and reorganized for readers.

The formulas have been revised and reorganized during the revision on this manuscript.

The conclusions should emphasize supervised learning, unsupervised learning, and deep learning in machine learning (ML) for network intrusion detection systems (NIDS).

We have updated the conclusion and emphasized each learning method for the NIDS.

The authors should discuss the future of the intrusion detection system and the future prospects for the cat and mouse game.

The revised conclusion and discussion sections now tackle these in a clear way.

Details:

Since this is a review paper, I hope the authors will correct the English and formulas themselves. Additionally, authors should revise the introduction and conclusion for the benefit of the readers.

Thank you for your valuable comments. We have addressed these comments and revised the manuscript accordingly. We proofread the entire manuscript and correct the formulas. We also revised the introduction and conclusion sections to benefit the readers.

Reviewer 4 Report

The review under consideration is generally of high quality and I can recommend it for publication provided that several points are addressed.

1. Tables 4 and 5 contain only general information about the surveyed studies but not their results. Summarizing the metrics reported in the reviewed works within these tables would improve the paper’s readability and usefulness.

2. Some formatting and presentation issues need correction. For example, the phrase “Type of the Paper (Review)” at the beginning of the manuscript should likely be simply “Review”. Many paragraphs in Section 4 have excessively large indentation, and there are cases where models and datasets are mentioned before they are first referenced (dataset references are given only in Section 3.1, yet many of those datasets are mentioned earlier). These and similar issues should be checked carefully.

3. In addition to the families of models considered, there are approaches such as AutoML-based models and others (see, for example, the recent review doi:10.3390/app151910389). It would be helpful to clarify the criteria used to select publications for inclusion in this review and to explain why particular model classes were considered.

Author Response

Reviewer 4:

The review under consideration is generally of high quality and I can recommend it for publication provided that several points are addressed.

Many thanks for your great comments.

Tables 4 and 5 contain only general information about the surveyed studies but not their results. Summarizing the metrics reported in the reviewed works within these tables would improve the paper’s readability and usefulness.

Thank you for your comments. We have carefully revised these Tables to improve the manuscript’s usefulness and readability significantly.

Some formatting and presentation issues need correction. For example, the phrase “Type of the Paper (Review)” at the beginning of the manuscript should likely be simply “Review”. Many paragraphs in Section 4 have excessively large indentation, and there are cases where models and datasets are mentioned before they are first referenced (dataset references are given only in Section 3.1, yet many of those datasets are mentioned earlier). These and similar issues should be checked carefully.

We have carefully checked and fixed these issues during the revision on the manuscript.

In addition to the families of models considered, there are approaches such as AutoML-based models and others (see, for example, the recent review doi:10.3390/app151910389). It would be helpful to clarify the criteria used to select publications for inclusion in this review and to explain why particular model classes were considered.

We clarified the critical use of some publications and mentioned the AutoML-based models in subsection 2.1.

Round 2

Reviewer 1 Report

The authors have well addressed my concerns.

n/a

Author Response

Thank you very much for your great comments.

Reviewer 2 Report

All comments have been addressed

Author Response

Thank you for your comments.

Reviewer 4 Report

he authors have made the necessary revisions and improved the presentation of the work. The article may be accepted for publication.

Some formatting issues and typographical errors may still be present in the text. For example, in several lines the word "where" likely should not be in italics; in line 291 the symbol hi should probably use a subscript, as in formula (6); in formula (7) the size of the parentheses should match the size of the fraction; in Section 4 the paragraph indentation is inconsistent, and so on. It is recommended to carefully proofread the manuscript before publication.

Author Response

We greatly appreciate your comments and feedback.

In several lines the word "where" likely should not be in italics.

We checked all the words “where” mentioned and made them non-italicized.

In line 291 the symbol hi should probably use a subscript, as in formula (6).

We changed hi to exactly reflect the one in formula (6).

In formula (7) the size of the parentheses should match the size of the fraction.

We changed the size of the parentheses to match the size of the fraction in formula (7).

In Section 4 the paragraph indentation is inconsistent, and so on.

We unified all the paragraph indentation to be consistent throughout the paper.

It is recommended to carefully proofread the manuscript before publication.

We have carefully proofread the entire manuscript and fixed the issues.