Federated Learning for IoT Intrusion Detection

Round 1

Reviewer 1 Report (Previous Reviewer 2)

The authors have improved the introduction as per my previous comments.

Author Response

Thanks for your feedback, I'm pleased to have addressed all of your previous comments.

Kind Regards

Riccardo Lazzarini

Reviewer 2 Report (New Reviewer)

1.     Abbreviations should be avoided in an abstract

2.     Name the aggregation algorithms you used in the abstract

3.     Add obtain results in the abstract

4.     No need to divide section 6 into two subsection

5.     Rename section 6 to Conclusions and future works

The English language's quality is excellent, and only minor editing is necessary. 

Author Response

Dear Reviewer,

Thanks for your feedback. Please find below my reply to your comments.

1. Abbreviations should be avoided in an abstract.

We have provided the expanded description for each of the acronyms used in the abstract - Some were not explained. I believe acronyms are used extensively in abstracts in many journals, including MDPI's. However, if you reckon that this should not be the case for this article I would be happy to change it.

2. Name the aggregation algorithms you used in the abstract.

We have included the names of the aggregation algorithms used as requested. That was clearly a mistake from my part.

3. Add obtain results in the abstract

Same as point 2. Comments about results in the abstract are now included.

4.  No need to divide section 6 into two subsection 

Concluding subsections were joined together into a single Section named "Conclusions and Future Works".

5. Rename section 6 to Conclusions and future works

See point 4. Concluding Section was renamed as requested.

I hope this addresses all of your comments satisfactorily.

Kind Regards,

Riccardo Lazzarini

Reviewer 3 Report (New Reviewer)

Authors evaluate the use of Federated Learning (FL) as a method to implement intrusion detection in IoT environments.

The research contribution of the paper is good, although there are some issues to fix in order to accept it:

In the related work section I find missing this related paper:

Conditional Variational Autoencoder for Prediction and Feature Recovery Applied to Intrusion Detection in IoT. Sensors 2017, 17, 1967.

After Figure 5 it is cited figure 11 instead of figure 6. Authors should check the numbering of the figures.

There is a title at the top of figures 5 to 10 that should be removed from the graph, and added at the bottom of each figure jointly with the figure number.

Authors should include their future work at the end of the conclusion section.

Author Response

Dear Reviewer,

Thanks for your feedback. Please find below my answers to your comments.

Point 1 - Suggested additional paper. 

I have included the article you have suggested. However, while it provides excellent insights in the area of IoT intrusion detection, it does not use FL as a method for the detection of anomalies. Therefore, I have included it at the end of the Section where I list several other relevant articles. I hope you find this acceptable and my answer to this point of your satisfaction.

Point 2 - Errors in Figures numbering

Thanks for spotting this. I have addressed this issue and it should now be corrected. The figures for the Confusion Matrix are in the latter pages of the article. That is why I mention Figure 11 or 12 in the same paragraph as Figure 5 or 6. These are used mainly for reference rather than providing comparative results. Having said that, the numbering I was referring to in the text were wrong and they are now corrected.

Point 3 - Removing titles on the Figures 5 to 10.

This has now been addressed. All titles were removed and the caption of each figures were expanded with those titles to provide more clarity. 

Point 4 - Adding Future Work in the concluding section.

This has now been addressed and reference to future works has been included.

Hope these answers address all of your point adequately.

Kind Regards

Riccardo Lazzarini

Reviewer 4 Report (New Reviewer)

This article presents an FL method that has the detection and classification of attacks in an IoT network environment as the primary objective compared to the use of a standard centralised approach. However, the results of the centralised models compared to FL models while taking into consideration the 'accuracy, precision, recall and F1-Score' are all very high with the use of both datasets, which doesn’t support the choice of FL except for the privacy side of the user.

·      The following sentence in line 5 expanding to line 6 doesn’t emphasise the concerns for the use centralised approach clearly which leads to thinking of adopting a decentralised approach.

·      The introduction has been written in a chronological way presenting IoT as a network then mentioning the IoT devices and their applications, then the machine learning techniques used to bring insight into the data shared between these devices, then deep learning techniques as part of ML, then a quick comparison between both techniques and the need to a centralised location. However, the limitations of centralised learning are summarised in two challenges such as a large amount of data to be transferred for training the model and the privacy, however, there are other limitations not mentioned.

·      In the implementation the choice to use a shallow ANN can be slightly explained or mentioned. The choice of open-source public datasets ToN_IoT and CI- CIDS2017 for the evaluation of the framework on both binary and multi-class classification should be justified.

·      More explanation is needed for the following questions: In line 262: How did you choose the number of rounds and why is the learning rate (LR) set to 0.01? A lack of explaining the empirical evaluation and why choosing the method Flower as a strategy?

·      In line 274: The reason for the choice of using the activation function ReLU is not mentioned.

·      Figure 3. can include more details of the shared model.

·      In line 430: The author doesn’t explain why the centralised model 'has clearly outperformed' while its accuracy is very high with a high precision rate which means few to no false positives compared to other FL models.

·      The conclusion doesn’t have to include all the previously explained details about federated learning, just touching on the importance of the use of FL and explaining the results.

The manuscript needs proof-reading especially since the authors have used a mix of US/UK writing in the same document, there are many examples in the document but for instance:

Line 126: Organization (US)Line 127: To analyse (UK) 

Author Response

Dear Reviewer,

Thanks for your feedback. Please find attached a pdf file containing answers to your comments. 

Kind Regards

Riccardo Lazzarini

Author Response File: Author Response.pdf

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.

Round 1

Reviewer 1 Report

strongnesses：

The paper proposed an scheme to use Federated Learning as a method to develop an engine for IDS in IoT environment, in which FL is based on four clients and one server, data analysis is performed at the client side.

Weaknesses:

1.The paper did not give and explain what the parameters sending to clients are.

2.The paper did not present the detailed method to aggregate and form the final results.

3.The paper did not provide the detailed formula to form the ML weights and model.

Reviewer 2 Report

The paper makes a long introduction of background but not clear on the issues and problem the authors would like to solve. Performance comparison with existing work is also not clear.

FL training data sets have many issues for example data correlation. C. Sun, S. Wu and T. Cui, "User Selection for Federated Learning in a Wireless Environment: A Process to Minimize the Negative Effect of Training Data Correlation and Improve Performance," in IEEE Vehicular Technology Magazine, vol. 17, no. 3, pp. 26-33, Sept. 2022, doi: 10.1109/MVT.2022.3153274.

The authors work is not rely on wireless network why the limited communication is mentioned in the literature review?

Reviewer 3 Report

The proposed an FL-based scheme to develop a mechanism to perform intrusion detection in IoT. The idea is interesting. However, I have the following concerns. 

1. Please revise the grammatical issues of the paper.  

2. Recent FL-based papers are missing. Some are mentioned as follows. 

-> "Federated Learning for Distributed IIoT Intrusion Detection using Transfer Approaches," in IEEE Transactions on Industrial Informatics, 2022, doi: 10.1109/TII.2022.3216575.

-> "FBI: A Federated Learning-Based Blockchain-Embedded Data Accumulation Scheme Using Drones for Internet of Things," in IEEE Wireless Communications Letters, vol. 11, no. 5, pp. 972-976, May 2022, doi: 10.1109/LWC.2022.3151873.

3. Contribution of this paper is limited. FL-based intrusion detection is a very popular topic and well-explored. Please highlight your novelty based on the limitation of existing works. 

4. Add a summary table for related works (i.e., Section 2) including the limitation of each.

5. Add input and output in algorithms.

6. A discussion on convergence is required. 

7. A Comparison with existing works is missing. 

8. Authors didn't consider the privacy of the model itself. 

9. A design of the DL model is required.