Addressing the Effectiveness of DDoS-Attack Detection Methods Based on the Clustering Method Using an Ensemble Method
Round 1
Reviewer 1 Report (Previous Reviewer 3)
In this paper, the authors focus on DDoS Attack Detection Methods based on the Clustering Method using an Ensemble Method. I suggest as follows:
1. The abstract is not convincing, it should be refined to precisely illustrate what authors have done in this paper and adding the major contribution of this work.
2. The author needs to clarify the new contribution of the research in the introduction and other section. It is necessary to clearly state the new and motivating points of the article and should be refined to precisely illustrate what authors have done in this paper.
3. In Section 3, I don't understand why you choose subsection 3.1 Introduction. I think that please write only the introduction of this section but not label 3.1. Please should begin subsection 3.1 DDoS Attacks.
4. In section 6 (Similar to Section 3), please write only the introduction of this section but not label 6.1.
5. The selected references are not enough to reveal the novel trend. The authors should add research papers published about DDoS Attack Detection Methods in international journals. Moreover, some recent references (paper in 2021-2022) to make the reference list self-sufficient.
6. Please compare the results of this paper and other recent papers about DDoS Attack Detection Methods in 2021-2022.
7. This paper should contain the significant advantage of this paper, please focus on it.
Author Response
Authors addressed the following comments of the first reviewer as follow:
- The abstract has been modified, refined, and addressed to reflect on reviewer’s comment.
- The contribution of the research in the introduction and other section have been modified to reflect on the reviewer comment.
- With respect to point 3 and 4 of reviewer comment, section 3 and section 6 have been modified addressed, and followed based on reviewer comments.
- With respect to points 5& 6 of reviewer comments previous studies have been added.
- With respect to point 7 the significant advantages which is the contribution of this study have been added.
Reviewer 2 Report (Previous Reviewer 2)
Dear Authors,
Well written paper with some simulated results but the paper can be improved by incorporating the following comments in the revised paper:
1. Section 2.1 and 2.2 are too small to be subsections. Advise to restructure it.
2. The content of 3.4 Section title and content does not match each other. Therefore, advise to restructure. The content of 3.4 section is not literature review. This is the process or steps you follow during the research.
3. Figure 1 is not cited from the text and lack of explanation of the Figure 1. Advise to add explanation of Figure 1.
4. Line number 310-311, the sentence meaning is not clear. Same applies to line 314- The table 3... is also not clear.
5. Pictorial comparison of the results from Table 3 would be nice for reader.
6. Line 376- "He" in the sentence does not sound good and later in the same paragraph.
Good luck.
Author Response
Authors addressed the following comments of the second reviewer as follow:
- With respect to point 1 we removed section 2.1 & 2.2 and restructured and incorporated the contents of 2.1&2.2 in the last paragraph of section 2 accordingly.
- With respect to point 2 of reviewer comment, we applied the CRISP-DM because it is a standard framework to address organizations problems. High false positive rate of DDoS attack detection methods in detecting attacks is considered an organizational problem. Further we elaborated more about this under section 3.4 for the use of CRISP-DM.
- With respect to point 3, 4,5, and 6 of reviewer comments respective clarifications are underscored within the research paper.
Reviewer 3 Report (Previous Reviewer 4)
The authors addressed my comments
Author Response
Comments were addressed.
Reviewer 4 Report (New Reviewer)
The paper is focused on determining whether adding the filter and wrapper methods preceded by combined clustering algorithms using the Vote classifier method was effective in lowering false positive rates of DDoS attack detection methods. So the topic is actual. The proposed cluster analysis method is adequate to solve the article a problem. The journal is correctly selected.
My recommendations to improve the paper as a major revision:
-> Abstract in present form is not well written. There is a lack of the main conclusion of this paper. Please extend.
-> line 28 should be revised "statistica1" should be "statistical"
-> please discuss the features of Anova method that makes it suitable to solve the problem with DDoS attack detection. maybe in line 222-223.
-> please extend the description of clustering methods with adequate equations to extend methods section.
-> in conclusions, the limitation of the proposed method should be added
-> in conclusions the future research directions are obligatory.
Author Response
Authors addressed the following comments of the Fourth reviewer as follow:
- With respect to point 1 of reviewer comment the abstract were addressed based reviewer recommendation.
- With respect to reviewer comment on line 28, it has been addressed.
- With respect to Anova and cluster methods, necessary information based on the Authors have been added to the paper.
- With respect to Limitation and future research in conclusion, they have been addressed and added to the conclusion.
Round 2
Reviewer 1 Report (Previous Reviewer 3)
This paper is scientifically valid and technically accurate in its methods and results.
I have checked the revised version. The authors have made all the required changes. I have no further comments and now I recommend this paper for acceptance to publication.
Reviewer 4 Report (New Reviewer)
After revision paper is suitable to publication.
This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.
Round 1
Reviewer 1 Report
There are so many cites to the authors dissertation "[1]" that w/o that document a full understanding of the methods is difficult at best.
The appendix which contains the actual results from experiments is very difficult to read and compare the results. The preference would be tables and graphs, not just tables.
This is a short paper, its doubtful the results could be reproduced because no flowchart or sequencing diagram shows the exact way (ordering of steps/methods, voting members, training vs testing, models) these experiments were conducted.
Organization is awkward. Usually the introduction explains the problem, motivation and leads naturally into background and related work. You don't explain how your problem statement (1.1) is addressed by your (3) research question and (4) Hypothesis. Thus, after your materials and methods (2) and all the sections, the reader finds that section (5) is the literature review. The grouping of your sections is confusing and non-standard. Some of the information you are describing in (5) sounds like it should be in (2) [e.g., data prep). You have an entire section that is one sentence (3). There is one section (4) that is two sentences. The last paragraph of (2) is written in the wrong tense (e.g., "Companies would be able to ...." this statement is not concrete as is true for most of the paragraph. You need to either remove this information or rewrite it so that it contributes to the understanding of what you accomplished in your research (e.g., you seem to be explaining the usefulness of CRIP-DM, but why tell us this in the materials and methods section? Rather, this is motivation and/or literature review. Oh, but the literature review is the methods....
What you accomplished is unclear. You have an Ho and an Ha. So you set up the experiments to prove Ho, but couldn't and therefore you proved Ha? That is a round about way to explain things. I think its important to focus on what worked, how and why, rather than explaining the how and why of what didn't work.
In conclusion, this is a very rough draft that is poorly organized and difficult to follow.
Author Response
The bolded areas here are the comments provided. The ones that are not in bold address the way the paper addresses the comments
Regarding "Are the methods adequately described."
This section was provided in a well-established manner. Section 2, which is Material and Methods, describes details of the methods. The first method is quantitative because we are evaluating false positive rates. We used ex post facto design of A-B-A-BC, because we wanted to build upon previous study conducted by Dr. Zeinalpour. All the dependent and independent variables are stated properly. And since this section of Material and Methods reflects on methods and materials, we reflected on two extra items of the CRISP-DM framework that we used as well as Weka on this section. The CRISP-DM is also further provided under literature review to reflect on its application.
Regarding "There are so many cites to the authors dissertation "[1]" that w/o that document a full understanding of the methods is difficult at best."
Under introduction of this paper, we provided a brief summary of the findings in dissertation "[1]" and the way it was conducted. The summary that we provided offers clear understanding of how [1] conducted the study and how the findings were described. Nonetheless we added more under the results section with the inclusion of the figure that if read can be understood and the results can be reproduced if followed.
Regarding "Are the results clearly presented"
We added more to the result section with inclusion of a figure that if followed the results can be reproduced.
Regarding "Are the conclusions supported by the results?"
We added more to the conclusion to associate the conclusion to the results with respect to addressing the effectiveness of DDoS attack detection methods.
Regarding Hypotheses
We used the ten-fold cross-validation method to either reject or accept the hypotheses. We were not able to verify or confirm based on the results. To add more, we did not need to conduct the statistical significance analysis. This is because the ten-fold cross-validation method provided in Weka already validated the results. We did not need to conduct statistical significance test analysis. Statistical significance test analysis is for census-based (survey-based) data that require some form of validation. The tables under appendices presents the full results. The summary of comparison between filter, wrapper, and combined clustering algorithms using vote classifier method is given under the results section of our paper using ex post facto design of A-B-A-BC.
Regarding organization of paper
We enhanced the organization of the paper following the 3rd reviewer comments.
Reviewer 2 Report
Dear Authors,
It is well written paper measure the effectiveness of DDoS attack detection methods. The problem formulation, research questions and hypothesis set very well. Literature review is done very nicely in three different relevant areas. However, advised to add introduction section or paragraph on Section 3, 4 and 5.
Section 6 of one paragraph does sound good and therefore, combine with Section 7 with necessary changes.
The paper presentation and readability can be improved in Section 8 by doing following changes.
- You have presented the outcome of the analysis in Appendices. It is better to present either in Tabular or bar chart forms in the section so that reader can grab the results of the experiment quickly and easily.
- Advised to put put introduction paragraph before 8.1 and conclusion paragraph at the end.
- Show your outcomes in clear statistical forms too for clarity.
There is too much reference to [1] in the paper which is unnecessary. Therefore, rewrite the content removing some in-text citations. In some places, it makes perfect sense but in some places it is unnecessary. For example, you can write the outcome of this paper as standalone without refereeing this paper. This same concept applied to many places.
Good luck.
Author Response
The bolded areas here are the comments provided. The ones that are not in bold address the way the paper addresses the comments
Regarding "Are the results clearly presented"
We added more to the result section with inclusion of a figure that if followed the results can be reproduced.
Regarding "Are the conclusions supported by the results?"
We added more to the conclusion to associate the conclusion to the results with respect to addressing the effectiveness of DDoS attack detection methods.
Regarding organization of paper
We enhanced the organization of the paper following the 3rd reviewer comments.
Reviewer 3 Report
The aim of this paper is to examine whether adding the filter and wrapper methods prior to the combined clustering algorithms using the Vote classifier method is effective in lowering false positive rates of DDoS attack detection methods. This paper is interesting. It can be published after revision. However, I suggest some comments as follows:
1. This paper has 11 Sections. I think that the authors reduce the number of sections, for example, section 2-4 should include only one section, section 3 (Research Question) and Section 4 (Hypotheses) should be subsections.
2. I don't understand the content of Section 5, Is it Literature Review ?
3. The authors should improve the results part. This section should be the main of this paper but it is not clear. Please explain it.
4. The Conclusions part : the authors should add the significant results and advantages of this study in paper.
Author Response
The bolded areas here are the comments provided. The ones that are not in bold address the way the paper addresses the comments
Regarding "Are the results clearly presented"
We added more to the result section with inclusion of a figure that if followed the results can be reproduced.
Regarding "Are the conclusions supported by the results?"
We added more to the conclusion to associate the conclusion to the results with respect to addressing the effectiveness of DDoS attack detection methods.
Regarding organization of paper
We enhanced the organization of the paper following your suggestions.
Reviewer 4 Report
- Abstract, for data mining framework, this --> for data mining framework. This …
- What are the limitations of related work? How will the authors address these limitation?
- What are the benefits vs costs when adding the filter and wrapper methods?
- Describe in more details the filter / wrapper methods used
- In chapter 8, “Based on the results of this study,”. The authors should conduct a proper statistical test before rejecting or not rejecting the null hypothesis
- Comparison with previous studies is not stated in details
- The contribution of the paper is still vague? What exactly is the contribution?
Author Response
The bolded areas here are the comments provided. The ones that are not in bold address the way the paper addresses the comments
Regarding "Are the methods adequately described."
This section was provided in a well-established manner. Section 2, which is Material and Methods, describes details of the methods. The first method is quantitative because we are evaluating false positive rates. We used ex post facto design of A-B-A-BC, because we wanted to build upon previous study conducted by Dr. Zeinalpour. All the dependent and independent variables are stated properly. And since this section of Material and Methods reflects on methods and materials, we reflected on two extra items of the CRISP-DM framework that we used as well as Weka on this section. The CRISP-DM is also further provided under literature review to reflect on its application.
Regarding "Are the results clearly presented"
We added more to the result section with inclusion of a figure that if followed the results can be reproduced.
Regarding Hypotheses and statistical significance testing
We used the ten-fold cross-validation method to either reject or accept the hypotheses. We were not able to verify or confirm based on the results. To add more, we did not need to conduct the statistical significance analysis. This is because the ten-fold cross-validation method provided in Weka already validated the results. We did not need to conduct statistical significance test analysis. Statistical significance test analysis is for census-based (survey-based) data that require some form of validation. The tables under appendices presents the full results. The summary of comparison between filter, wrapper, and combined clustering algorithms using vote classifier method is given under the results section of our paper using ex post facto design of A-B-A-BC.
Regarding contribution of the study
The contribution of the study is placed in the last statemen of the "Abstract".
Round 2
Reviewer 1 Report
In Sect "Purpose and Hypotheses of the Study" you provide two Hypothesis. Why not tell us at the end of this paragraph the result immediately? Its appropriate to give the results immediately, and allow the reader to read on to see if they agree.
You need to add a histogram(s) that shows the results of your study organized in a fashion that corresponds to your flow chart.
Your tables at the end of the paper located prior to the bibliography are labeled incorrectly: "Appendix A: Independent Variables Table" a table is a table, it's not an appendix! An appendix is a section placed after the bibliography. Create an Appendix A: Experimental Results. Then explain in the text what experimental results are enumerated in which table 1, 2, 3, ... Provide some explanation of why these tables are being provided and link those results back to the histograms that you have newly created in the results section above.
Those tables are necessary but not sufficient for a complete study. You need some visual graphic (e.g., histogram) so we can actually visually compare the results across the board of the different experiments so that its clear weather your experiments proved/disproved the Hypothesis.
You must further address your hypothesis testing in the conclusions. What conclusions do you draw with respect to your hypotheses and why? You say "lowest" FP rate is achieved, but compared to what? You say Using the Vote classifier method may be considered an advantage..." because why? and compared to what? You are asking your readers to plow through this paper and at the end, you need to say something more concrete like, "...method should be considered..." because of the results we are providing in our study and the hypothesis testing we accomplished.
Author Response
Greetings,
The bold texts below present the comments that the reviewer provided.
Reviewer Comment
In Sect "Purpose and Hypotheses of the Study" you provide two Hypothesis. Why not tell us at the end of this paragraph the result immediately? Its appropriate to give the results immediately, and allow the reader to read on to see if they agree.
Our Answer
The purpose of our study was to determine whether adding the filter and wrapper methods prior to the combined clustering algorithms using the Vote classifier was effective in lowering false positive rates of DDoS attack detection methods. Just as a note, this is one and only one purpose. We were not able to verify the effectiveness based on the results. This is because it was not in both cases; that is when we incorporated both filter and wrapper methods. This was with respect to the corresponding clustering algorithms’ integrations as well as comparison across the rest of the experimentation results in this study. However, we added an explanation of the end determination with respect to research question and its respective hypotheses corresponding to what the reviewer nearly wanted in respect to the end determination.
Reviewer Comment
Regarding leaving the decision to readers to decide the effectiveness.
Our Answer
Thank you for your comment. With respect to your approach that we will consider in a different research study. Please note, the purpose of this research that we conducted is strictly to get the results based on the purpose of the study, and we like to inform the readers of the actual results.
Reviewer Comment
Reviewer comment regarding tables and graphs
Our Answer
We already provided tables that show the results and the difference. The figure shows the way we conducted the study for reproducing the results. Dr. Zeinalpour's dissertation was conducted in a similar way without providing graphs. We followed similar way. The goal of our research was to determine whether adding the filter and wrapper methods was effective in lowering false positive rates of DDoS attack detection methods; more importantly this is one and only one purpose. This is because it was not in both cases; that is when we incorporated both filter and wrapper methods
The experimentation was done with consideration of what was appropriate to get the result. , secondly. We were happy, excited and pleased to read that one reviewer of this paper in this journal stated that this paper is now ready for publication. Nevertheless, we followed your recommendation and cleaned the explanation of the results a little bit to be more clear with respect to your last review.
Reviewer Comment
Regarding reviewer’s comment on inadequateness of the explanation of methods and material.
Our Answer
We explained every method and material in their essence. They are fully described. Besides, two reviewers of this paper agree with the way we explained the method and design of the study. One reviewer stated that this paper is now ready for publication.
Reviewer Comment
Those tables are necessary but not sufficient for a complete study. You need some visual graphic (e.g., histogram) so we can actually visually compare the results across the board of the different experiments so that its clear weather your experiments proved/disproved the Hypothesis.
Our Answer
Firstly, one reviewer stated that this paper is now ready for publication. Secondly, we already finished the experimentation. Thirdly, please refer to the purpose statement of our study which reflects on that we only have one purpose in our study, and therefore its respective null and alternative hypotheses. The purpose of our study was to determine whether adding the filter and wrapper methods prior to the combined clustering algorithms using the Vote classifier method was effective. Based on the results, we cannot verify whether adding filter and wrapper methods prior to the combined clustering algorithms using the Vote classifier is effective. This is because it was not in both cases; that is when we incorporated both filter and wrapper methods.
Reviewer Comment
Regarding reviewer’s comment on conclusion:
Our Answer
We enhanced it based on the reviewer’s recommendations.
Reviewer Comment
Regarding placement of the appendices after references:
Our Answer
We followed the electronics template.
Reviewer 3 Report
Now, this paper is well revised according to all reviewer's point out. I decide that it can be published.
Author Response
We appreciate reviewer's comment in deciding that our paper is good for publication.
Reviewer 4 Report
A statistical test must be conducted to confirm the significance of the results. If model A has a higher accuracy (or less error) than model B, this does not mean that model A is better.
Authors are advised to use parametric on non-parametric (depends which one is appropriate) to check whether the results are significant or not
Author Response
Greetings,
The bold texts below present the comments that the reviewer provided.
Reviewer’s comments:
A statistical test must be conducted to confirm the significance of the results. If model A has a higher accuracy (or less error) than model B, this does not mean that model A is better.
Authors are advised to use parametric on non-parametric (depends which one is appropriate) to check whether the results are significant or not.
Our Answer
Statistical significance testing is for survey-based data. Statistical significance testing does not fit with the purpose of our study to determine the significance. This is because our purpose of study does not ask for the significance of difference. The purpose of our study was to determine whether adding the filter and wrapper methods prior to the combined clustering algorithms using the Vote classifier was effective in lowering false positive rates of DDoS attack detection methods. Just as a note, this is one and only one purpose. We were not able to verify the effectiveness based on the results. This is because it was not in both cases; that is when we incorporated both filter and wrapper methods.
Round 3
Reviewer 4 Report
The authors refused to conduct a statistical test to confirm if the results are significant or not. His response is not accurate.
So based on this, I cannot confirm if the results are statistically significant.
