An Efficient CNN-Based Intrusion Detection System for IoT: Use Case Towards Cybersecurity

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The authors propose a deep learning framework that incorporates several optimizations, including dimensionality reduction, hyper-parameter tuning, and feature engineering, aimed at detecting and classifying cyberattacks. Central to this framework is the Intelligent Intrusion Detection Network (IIDNet), an enhanced Convolutional Neural Network (CNN) variant designed to improve detection performance. The study is evaluated using the UNSW-NB15 dataset.

While the work presents an interesting approach, I have a major concern regarding the paper’s treatment of time complexity.

The paper falls within the extensive body of literature that applies ML/DL-based techniques to network intrusion detection. However, like many similar works, it overlooks the time complexity associated with deep learning methods, particularly CNNs, where convolution operations can be computationally expensive. This omission is problematic, as time complexity is a crucial factor when detecting cyberattacks, where rapid detection is often essential.

I strongly recommend that the authors place more emphasis on comparing their approach with related works that account for the time complexity of the employed deep learning strategies. Unfortunately, much of the recent literature, including this paper, tends to focus solely on classic performance indicators such as accuracy and F-measure, while neglecting the critical aspect of time complexity, which is essential for ensuring timely cyberattack detection.

I suggest performing a comparison with the following related works, where time complexity is given significant consideration:

"Network Abnormal Traffic Detection Model Based on Semi-Supervised Deep Reinforcement Learning," IEEE Transactions on Network and Service Management, 2021;

"Experimental Review of Neural-Based Approaches for Network Intrusion Management," IEEE Transactions on Network and Service Management, 2020;

"Systematic Approach to Analyze The Avast IOT-23 Challenge Dataset For Malware Detection Using Machine Learning," 18th International Conference on Emerging Technologies, 2023.

Finally, several figures (e.g., figures 5, 9, and 11) are difficult to read due to their low resolution and should be improved for clarity.

Author Response

Comment 1:
The authors propose a deep learning framework that incorporates several optimizations, including dimensionality reduction, hyper-parameter tuning, and feature engineering, aimed at detecting and classifying cyberattacks. Central to this framework is the Intelligent Intrusion Detection Network (IIDNet), an enhanced Convolutional Neural Network (CNN) variant designed to improve detection performance. The study is evaluated using the UNSW-NB15 dataset.

• Response 1:
  We thank the reviewer for recognizing the contributions of our proposed framework, including the key optimizations that improve the performance of IIDNet for cyberattack detection. We have further refined the description of these methods in the manuscript to ensure clarity and comprehensiveness.

 

Comment 2:
While the work presents an interesting approach, I have a major concern regarding the paper’s treatment of time complexity.

• Response 2:
  We appreciate the reviewer’s feedback regarding the time complexity of the proposed IIDNet model. In response to this, we have added a detailed section on Time Complexity (Section 4.6.1) in the revised manuscript. This section provides an in-depth analysis of the computational complexity of the convolution operations in the CNN layers and how dimensionality reduction and hyperparameter tuning contribute to optimizing the model's time complexity. Additionally, empirical results on training and inference time, measured in a GPU-powered environment, are also provided for comparison with related models.

 

Comment 3:
The paper falls within the extensive body of literature that applies ML/DL-based techniques to network intrusion detection. However, like many similar works, it overlooks the time complexity associated with deep learning methods, particularly CNNs, where convolution operations can be computationally expensive. This omission is problematic, as time complexity is a crucial factor when detecting cyberattacks, where rapid detection is often essential.

• Response 3:
  Thank you for this suggestion. In light of this comment, we have revised the manuscript to explicitly address the computational complexity of CNNs, especially in real-time applications. We have provided an in-depth analysis of the convolutional layer time complexity and discussed how the proposed IIDNet framework leverages optimizations such as PCA and t-SNE to mitigate computational costs. These enhancements allow IIDNet to be effective even in environments requiring rapid cyberattack detection, thus aligning with real-world constraints.

 

Comment 4:
I strongly recommend that the authors place more emphasis on comparing their approach with related works that account for the time complexity of the employed deep learning strategies. Unfortunately, much of the recent literature, including this paper, tends to focus solely on classic performance indicators such as accuracy and F-measure, while neglecting the critical aspect of time complexity, which is essential for ensuring timely cyberattack detection.

• Response 4:
  Thank you for this valuable suggestion. We have revised the manuscript to include a comparison with relevant literature that addresses time complexity in deep learning-based intrusion detection systems. In particular, we have discussed and compared IIDNet's performance against the models presented in the following works:
• "Network Abnormal Traffic Detection Model Based on Semi-Supervised Deep Reinforcement Learning" (IEEE Transactions on Network and Service Management, 2021)​(MDPI_CNN_Article_latest).
• "Experimental Review of Neural-Based Approaches for Network Intrusion Management" (IEEE Transactions on Network and Service Management, 2020)​(MDPI_CNN_Article_latest).
• "Systematic Approach to Analyze The Avast IOT-23 Challenge Dataset For Malware Detection Using Machine Learning" (2023)​(MDPI_CNN_Article_latest).

 

Comment 5:
I suggest performing a comparison with the following related works, where time complexity is given significant consideration:

• Response 5:
  Thank you for the valuable suggestion. As recommended, we have compared our approach with the related works you suggested:
1. "Network Abnormal Traffic Detection Model Based on Semi-Supervised Deep Reinforcement Learning" (2021)​(MDPI_CNN_Article_latest).
2. "Experimental Review of Neural-Based Approaches for Network Intrusion Management" (2020)​(MDPI_CNN_Article_latest).
3. "Systematic Approach to Analyze The Avast IOT-23 Challenge Dataset For Malware Detection" (2023)​(MDPI_CNN_Article_latest).

In the revised manuscript, these works are discussed in terms of their treatment of time complexity and how IIDNet’s optimizations address the same challenges. We have outlined the strengths and weaknesses of each approach relative to IIDNet, particularly in real-time intrusion detection scenarios.

Comment 6:
Finally, several figures (e.g., figures 5, 9, and 11) are difficult to read due to their low resolution and should be improved for clarity.

• Response 6:
  Thank you for the suggestion. We acknowledge the issue with figure resolution. We have replaced Figures 5, 9, and 11 with high-resolution versions to enhance clarity and readability in the revised manuscript. The figures are now more detailed and legible, ensuring that the visual representation of results is clear to the readers.

Reviewer 2 Report

Comments and Suggestions for Authors

Review comments:

1. The paper is well-structured and introduces intrusion detection systems (IDS) in the Internet of Things (IoT) environment and solutions based on deep learning. The article has clear logic from introduction, related work, methodology, experimental results to discussion, conclusion and future work.

- **Comprehensive content**: It covers the main challenges in the current IoT network security field and proposes an intrusion detection system (IIDNet) based on convolutional neural network (CNN). The article also discusses technical optimizations such as data dimension reduction, feature selection, and hyperparameter tuning. These contents are rich and detailed, suitable for technical readers.

 

2. The article effectively points out the security challenges in the IoT environment in the introduction and explains the importance of artificial intelligence and deep learning in network security. This part lays a good foundation for subsequent research. The problem definition part points out the limitations of traditional IDS when dealing with large-scale IoT devices. The problems mentioned in the article, such as high-dimensional data and resource constraints, provide a reasonable background for proposing new solutions.

 

3. An improved CNN architecture (IIDNet) is proposed, and how to improve model performance through layer optimization, hyperparameter tuning and other methods is discussed in detail. This part is detailed and the technical details are clear. The article lists the detailed process of hyperparameter optimization through GridSearchCV, which is an effective optimization strategy. Combining classic dimensionality reduction methods such as PCA and t-SNE can help reduce the dimension of the data set and improve the running efficiency of the model. By using XGBoost for feature selection, this combination method enhances the effectiveness of the model.

 

4. Experimental and result analysis, experiments were conducted using the UNSW-NB15 dataset, which is a commonly used dataset in the field of intrusion detection. The article shows the detailed process of data preprocessing, model training, and performance evaluation. The experimental results show that the model shows high accuracy (95.47%) when dealing with different types of attacks. The robustness of the model in classifying attacks and normal traffic is verified by the analysis of the confusion matrix. Comparative experiments with other models (such as MLP and Baseline CNN) further prove the effectiveness of the IIDNet model, indicating that the model outperforms other methods in multiple indicators.

 

5. The article mentioned in the discussion section that this method is only verified based on the UNSW-NB15 dataset, and real network traffic data should be used for testing in the future. At the same time, no hybrid technology is used in the feature selection process, which is a reasonable direction for improvement.

 

Suggestions for improvement:

Although the article has shown the performance comparison with other models, a more detailed comparative analysis can be added to explain why IIDNet is superior to other models in some aspects. The complexity and training time of IIDNet are not discussed in detail. It is recommended to add analysis on the training time and resource usage of the model under different data scales, which will help readers understand the actual application of the model. 

Summary:

This paper provides an innovative CNN-based IoT intrusion detection system and verifies its effectiveness through experiments. The overall structure of the paper is reasonable, the content is detailed, and it is suitable for publication. Further improvements can focus on introducing more real-world data sets for testing and analyzing system resource consumption.

Comments on the Quality of English Language

More transition sentences can be added in some paragraphs to make the connection between different parts smoother.

 

Author Response

Comment 1:
The paper is well-structured and introduces intrusion detection systems (IDS) in the Internet of Things (IoT) environment and solutions based on deep learning. The article has clear logic from introduction, related work, methodology, experimental results to discussion, conclusion and future work.

• Response 1:
  Thank you for your positive feedback on the structure and logical flow of the paper. We have ensured that each section flows logically to present a comprehensive understanding of the research. The introduction and subsequent sections now have minor improvements in clarity and explanation.

Comment 2:
The article effectively points out the security challenges in the IoT environment in the introduction and explains the importance of artificial intelligence and deep learning in network security. This part lays a good foundation for subsequent research. The problem definition part points out the limitations of traditional IDS when dealing with large-scale IoT devices. The problems mentioned in the article, such as high-dimensional data and resource constraints, provide a reasonable background for proposing new solutions.

• Response 2:
  We appreciate your recognition of the comprehensive background and problem definition. We further enhanced the explanation of the key challenges, particularly the discussion on resource constraints and high-dimensional data, to better set the context for the proposed deep learning solution (IIDNet).

Comment 3:
An improved CNN architecture (IIDNet) is proposed, and how to improve model performance through layer optimization, hyperparameter tuning and other methods is discussed in detail. This part is detailed and the technical details are clear. The article lists the detailed process of hyperparameter optimization through GridSearchCV, which is an effective optimization strategy.

• Response 3:
  Thank you for acknowledging the detailed description of the CNN architecture and optimization techniques. We have slightly refined the explanations regarding hyperparameter tuning and layer optimization, ensuring that the technical processes are even more precise and accessible to readers.

Comment 4:
Experimental and result analysis, experiments were conducted using the UNSW-NB15 dataset, which is a commonly used dataset in the field of intrusion detection. The article shows the detailed process of data preprocessing, model training, and performance evaluation. The experimental results show that the model shows high accuracy (95.47%) when dealing with different types of attacks. The robustness of the model in classifying attacks and normal traffic is verified by the analysis of the confusion matrix. Comparative experiments with other models (such as MLP and Baseline CNN) further prove the effectiveness of the IIDNet model, indicating that the model outperforms other methods in multiple indicators.

• Response 4:
  We are grateful for your positive feedback on the experimental design and results. To further emphasize IIDNet’s superiority, we have provided additional details regarding its comparative advantage over the other models, including a more in-depth discussion on its ability to handle imbalanced data, as showcased by the confusion matrix and performance metrics.

Comment 5:
The article mentioned in the discussion section that this method is only verified based on the UNSW-NB15 dataset, and real network traffic data should be used for testing in the future. At the same time, no hybrid technology is used in the feature selection process, which is a reasonable direction for improvement.

• Response 5:
  We fully agree with your suggestion regarding the limitations of using a single dataset (UNSW-NB15). In the revised manuscript, we have expanded the Discussion section to highlight the potential for future work involving real-world network traffic and hybrid feature selection techniques. We also emphasized how hybrid methods could enhance the robustness and adaptability of IIDNet in various contexts.

Suggestions for Improvement:

Comment 6:
Although the article has shown the performance comparison with other models, a more detailed comparative analysis can be added to explain why IIDNet is superior to other models in some aspects. The complexity and training time of IIDNet are not discussed in detail. It is recommended to add analysis on the training time and resource usage of the model under different data scales, which will help readers understand the actual application of the model.

• Response 6:
  We appreciate your constructive suggestion. In response, we have expanded the performance comparison section to provide a more detailed explanation of why IIDNet outperforms other models, specifically in terms of precision, recall, and overall robustness against different attack types. Additionally, we have added a section on Training Time Analysis with Different Data Scales, which includes the model’s performance under varying dataset sizes, resource usage (GPU memory, CPU utilization), and computational complexity. This will help readers better understand the practical implications of applying IIDNet in real-world scenarios.

Comment 7:
More transition sentences can be added in some paragraphs to make the connection between different parts smoother.

• Response 7:
  In line with your suggestion, we have revised various sections throughout the manuscript to add smoother transitions between paragraphs, improving the readability and flow of ideas. We have added contribution statements, dataset in more detail, discussion section in more detail etc. Ensuring that the connections between different parts of the paper are clearer and more coherent for the readers.

Comment on English Quality:
Minor editing of English language required.

• Response on English Quality:
  Thank you for your feedback on the language. We have carefully reviewed the manuscript and made minor edits to improve clarity, readability, and grammatical accuracy. We have removed inverted commas in section 3.1, improved clarity of sentences etc.

Reviewer 3 Report

Comments and Suggestions for Authors

The references need to be reformatted to appear in a proper sequence: 1, 2, 3, 4, 5, 6, 7….. Otherwise it looks untidy.

Page 1 line 22: It is good to write a word in full “it is” and avoid abbreviations.

Page 2 lines 58 and 59: “Numerous researchers have contributed to developing using deep learning techniques for intrusion detection applications”. The sentence is not quite clear. Please review it.

Reading through the paper a number of sentences are not clear and do not seem to explain matters or extend the logic of what is being said. The writing needs to have greater clarity throughout the paper. This means sentences have to be accurate and complete.

Page 2 line 74: which survey? What is being referred to?

It is not always easy to follow the flow of  argument or understand what is being reported. The context throughout needs to be clearer and so does the signposting.

Section 3.1: why is the sentence in inverted commas? Is it a quotation?

Why in Figure 1 does it state: “deep learning waste framework”…? This needs to be clarified.

More emphasis could be provided in terms of training data and training the model.

Page 6 lines 237 and 238 : “To choose the most crucial characteristics for training, embedded techniques like decision trees and random forests use feature selection in the model-building phase”. This can be explained further.

Page 8 Section 3.8: more information needs to be provided as regards the data set itself.

Page 11: reference can be made to the quality of the data training at the start. What has emerged from it? What needs to be taken into consideration? And why is this the case?

The Discussion section needs to be expanded. What other factors or considerations can be included to bring out the uniqueness of the findings?

 Please check the accuracy, completeness and uniformity of the references in the reference section.

 

 

 

 

Comments on the Quality of English Language

The references need to be reformatted to appear in a proper sequence: 1, 2, 3, 4, 5, 6, 7….. Otherwise it looks untidy.

Page 1 line 22: It is good to write a word in full “it is” and avoid abbreviations.

Page 2 lines 58 and 59: “Numerous researchers have contributed to developing using deep learning techniques for intrusion detection applications”. The sentence is not quite clear. Please review it.

Reading through the paper a number of sentences are not clear and do not seem to explain matters or extend the logic of what is being said. The writing needs to have greater clarity throughout the paper. This means sentences have to be accurate and complete.

It is not always easy to follow the flow of  argument or understand what is being reported. The context throughout needs to be clearer and so does the signposting.

Section 3.1: why is the sentence in inverted commas? Is it a quotation?

 

 

Author Response

Comment 1 (Page 1, line 22):
It is good to write a word in full “it is” and avoid abbreviations.

• Response 1:
  Thank you for your valuable comment. We appreciate the reviewer’s suggestion regarding the use of formal writing. In response, we have revised the sentence in Page 1, line 22 by replacing "it’s" with "it is" to ensure consistency in formal tone.

Comment 2 (Page 2, lines 58 and 59):
"Numerous researchers have contributed to developing using deep learning techniques for intrusion detection applications". The sentence is not quite clear. Please review it.

• Response 2:
  Thank you for your valuable comment. We agree with the reviewer that the original sentence lacked clarity. The sentence has been revised to:
  "Numerous researchers have contributed to the development of deep learning techniques for intrusion detection applications."
  This revision improves clarity and better conveys the intended meaning.

Comment 3 (Page 2, line 74):
Which survey? What is being referred to?

• Response 3:
  We have clarified the reference to the survey in Page 2, line 74. The revision now specifies the context of the survey and includes a citation where appropriate. This ensures that the reader understands the reference and the context of the survey.

Comment 4 (Flow and clarity throughout the paper):
The writing needs to have greater clarity throughout the paper. This means sentences have to be accurate and complete. It is not always easy to follow the flow of argument or understand what is being reported. The context throughout needs to be clearer and so does the signposting.

• Response 4:
  Thank you for your valuable comment. We have carefully reviewed the manuscript to improve the clarity of sentences, ensuring that arguments flow logically and coherently. Several long sentences were broken down into shorter ones to enhance readability. Additionally, we rephrased ambiguous statements and added transition sentences between sections to improve the overall flow of the paper.

 

Comment 5 (Section 3.1):
Why is the sentence in inverted commas? Is it a quotation?

• Response 5:
  Thank you for your valuable comment. We appreciate the reviewer’s observation. The sentence in Section 3.1 was not a direct quotation. Therefore, we have removed the inverted commas to reflect this and maintain consistency in style.

Comment 6 (Figure 1):
Why does it state: "deep learning waste framework"? This needs to be clarified.

• Response 6:
  Thank you for your valuable comment and pointing out this issue. The term "deep learning waste framework" was a typographical error. It has been corrected to "deep learning framework" in Figure 1 for clarity.

Comment 7 (Page 6, lines 237 and 238):
"To choose the most crucial characteristics for training, embedded techniques like decision trees and random forests use feature selection in the model-building phase." This can be explained further.

• Response 7:
  Thank you for your valuable comment. In response to this comment, we have expanded the explanation of how decision trees and random forests use embedded feature selection during model construction. The revised text provides more detail on how these methods evaluate the importance of features to enhance model accuracy and efficiency.

Comment 8 (Page 8, Section 3.8):
More information needs to be provided regarding the dataset itself.

• Response 8:
  Thank you for your valuable comment. We have added additional details about the UNSW-NB15 dataset in Section 3.8, including its features, characteristics, and relevance in intrusion detection research. Specifically, we mentioned how this dataset simulates real-world network traffic and attacks, which makes it suitable for evaluating the effectiveness of our proposed IIDNet model.

Comment 9 (Page 11):
Reference can be made to the quality of the data training at the start. What has emerged from it? What needs to be taken into consideration?

• Response 9:
  Thank you for your valuable comment. We have added a discussion on the quality of the training data in Page 11, elaborating on its importance in model performance. We addressed how preprocessing steps such as normalization and feature scaling improve the quality of the dataset and reduce noise, which in turn enhances the model’s ability to detect intrusions accurately.

Comment 10 (Discussion Section):
The Discussion section needs to be expanded. What other factors or considerations can be included to bring out the uniqueness of the findings?

• Response 10:
  In response to this suggestion, we have significantly expanded the Discussion Section to highlight the robustness of IIDNet in generalizing across different types of attacks. We have also emphasized the importance of feature selection and hyperparameter tuning in optimizing the model’s performance. Additionally, we mentioned possible future work, including hybrid approaches and real-time data collection, to further enhance the detection capabilities of IIDNet.

Comment 11 (References Section):
Ensure all references are uniform and correct in their format. Check for missing details such as page numbers or DOI. Correct any inconsistencies.

• Response 11:
  We have thoroughly reviewed the References Section to ensure uniformity and correctness in formatting. Missing details, such as page numbers and DOI links, have been added where necessary. Any inconsistencies in citation style have also been corrected, and the references are now listed in proper sequential order.

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

The authors have made a commendable effort to address all the comments I raised in the previous stage of review. In particular they have:

- Better investigated the problem related to the time complexity that, when using DL-based methods, cannot be neglected;

- Considered comparisons with recent and credited works which focus on temporal complexity of DL methods when applied to the IDS field;

- Improved the quality of figures.

In my opinion, the paper can be now accepted in its current form.

Reviewer 3 Report

Comments and Suggestions for Authors

The manuscript is fine.