Analysis of Consumer IoT Device Vulnerability Quantification Frameworks
Round 1
Reviewer 1 Report
Comments for author File: 
Comments.pdf
Author Response
Reviewer comment: Overall, the article is well written although I do suggest that it be sent for language editing. I also see some repetition here and there. Specifically, where the significance of this review is argued. It is nicely argued in the introduction and then appears again in the next section after Table 1. Please see the rest of my comments below.
Our response: We would like to thank you for your suggestions and comments. They are greatly appreciated.
Reviewer comment: Abstract: The abstract is clear and well written. Introduction: This is a nicely written introduction which clearly argues the rationale for this review.
Our response: Thank you very much again for the positive comment.
Reviewer comment: Related work: I think that the figure needs to be redrawn as their text is somewhat pixelated. I would reword the text in Figure 1 on the right. For example, the use of “luck” does not fit in my opinion. If the figures are adapted from known sources, please acknowledge those sources in the caption. This applies specifically to Figure 4.
Our response: We have redrawn the figure and revised the text and reworded them as suggested.
Reviewer comment: Additionally, this section defines an IoT vulnerability for the first time. I suggest defining this in the introduction as well.
Our response: We have moved the definition of IoT vulnerability to the introduction section (second paragraph) and highlighted the statement in yellow.
Reviewer comment: Methodological aspects: Should the second research question not use the word frameworks instead of approaches? I understand this review to be focused on the frameworks that could be used.
Our response: We have updated it as suggested.
Reviewer comment: Although I value to creativity used to develop Figure 5, I suggest the use of a more traditional PRISMA diagram so that one can see all the search results and not just those related to the database searching. The rest of the methodology is clear, and I can see the inclusion and exclusion criteria including the quality assessment process. When I perform a review like this, I usually develop a data mapping or extraction table which shows the article synthesis based on the key aspects considered. Was this completed for the final set of review articles? Also, I don’t see the final set of review articles. All the above needs to feature clearly in a systematic review.
Our response: We have added section “4.3 Data extraction and synthesis” including a data mapping or extraction table which shows the article synthesis based on the key aspects considered. We have also replaced Fig. 5 with the traditional PRISMA diagram as suggested. For the sake of brevity, we have included the final set of review articles in Table 5 un-der “References” column.
Reviewer comment: Results and discussion: I like how each research question is discussed in its own subsection. However, I would like to suggest that some more critical insight be provided at the end of each subsection (i.e., 5.1 and 5.2 etc.). What does this mean for the study of IoT vulnerability assessment going forward?
Our response: We have added more critical insight at the end of each 5.1 and 5.2 subsections.
Reviewer comment: In 5.2 the word severity is added to the mix, which is not mentioned when the research questions are first introduced. Please align these aspects.
Our response: We have replaced “vulnerability severity” with “vulnerability score”.
Reviewer comment: The depth of 5.1 is great, however I do not see the critical insight here. For example, it would be interesting to know which of these assessment frameworks are used most or perhaps rank them. Doing this on a per industry basis would be even more valuable. I know the focus was on identifying them, but this would be a valuable addition and useful for future researchers, which is the purpose of such a review. The same applies to 5.3 where it would be very interesting to see where these frameworks are used most. I can see the number of references there, but perhaps a visualisation of some kind could be developed to clearly illustrated those used most. The two blue hues used in Figure 8 are too similar. Please use different colours here.
Our response: We have added critical insight as suggested. Furthermore, Fig.7 along explanation have been added to show the distribution of the frameworks ranked in percentile as suggested.
Reviewer comment: Conclusions: A much stronger emphasis should be placed on the future research in this area. What are the research implications of the findings. This is what a future researcher needs to look at when they wish to choose a way forward. It should be rich with advice and recommendations. Essentially, if I were to give this paper to a potential PhD student would they be able to use it to select a novel study approach? This is what I try to encapsulate in my reviews.
Our response: We have rewritten the conclusion and the future research in the area have been emphasised.
Reviewer 2 Report
1) Frequent repetition of expressions, for example 161-163 "vulnerable device".
2)Figure 4 does not refer to the chapter in which it is located.
3) Paragraph 3.2 is missing
4) Frequent consecutive repetition of the same conclusions
5) There is a lack of specifics in describing the impact of threats on the Internet of Things in various fields
6) It is worth separating the description of each level of the architecture of the Internet of Things , complementing the threats associated specifically with this level
Author Response
We would like to thank you for your suggestions and comments. They are greatly appreciated.
Reviewer comment: Frequent repetition of expressions, for example 161-163 "vulnerable device".
Our response: Thank you for identifying this mistake. We have corrected it. This statement now reads “Once such vulnerable devices have been identified, a malicious payload or command is executed in the device.”
Reviewer comment: Figure 4 does not refer to the chapter in which it is located.
Our response: Figure 4 does appear in the text. It appears in the statement “The systematic methodology is composed of three phases as outlined in Fig. 4” in the first paragraph of section 4.
Reviewer comment: Paragraph 3.2 is missing
Our response: Thank you for identifying this mistake. We have corrected it.
Reviewer comment: Frequent consecutive repetition of the same conclusions
Our response: We have proofread the document and have corrected all we have come across.
Reviewer comment: There is a lack of specifics in describing the impact of threats on the Internet of Things in various fields.
Our response: We appreciate the comment. Our work is focused on SLR of IoT vulnerability assessment frameworks. Therefore, “describing the impact of threats on the Internet of Things in various fields" is not within the scope of the work we presented in the paper.
Reviewer comment: It is worth separating the description of each level of the architecture of the Internet of Things, complementing the threats associated specifically with this level.
Our response: As you suggested, we have organised the security vulnerabilities according to each layer of the IoT architecture as follows:
3.1.1 Physical layer
3.1.2 Network layer
3.1.3 Application layer
Reviewer 3 Report
Strong aspects:
This paper reviews the work done on exploring the vulnerabilities of the IoT and highlights the importance of this field of research which less explored.
Weak aspects:
The paper represents an interesting discussion on the topic. However, some new technical methods for increasing of the security of IoT should be discussed. For example, in the authors opinion how the security of the systems can be evaluated ? In the case of vulnerabilities how the system can be improved ?
Typos:
line 532 - 'methods are practiced to demonstrate the practicality' should be revised to avoid repetition
line 538 - In conclusions: alarming
Author Response
Reviewer comment: Strong aspects: This paper reviews the work done on exploring the vulnerabilities of the IoT and highlights the importance of this field of research which less explored.
Our response: Thank you very much for the positive comment.
Reviewer comment: The paper represents an interesting discussion on the topic. However, some new technical methods for increasing of the security of IoT should be discussed. For example, in the authors opinion how the security of the systems can be evaluated ? In the case of vulnerabilities how the system can be improved ?
Our response: This are good suggestion for survey papers. Our work is focused on SLR of IoT vulnerability assessment frameworks. Therefore, “describing how the security of the systems can be evaluated? In the case of vulnerabilities how the system can be improved?" is not within the scope of the work we presented in the paper.
Reviewer comment: line 532 - 'methods are practiced to demonstrate the practicality' should be revised to avoid repetition
Our response: We have revised it as suggested.
Reviewer comment: line 538 - In conclusions: alarming
Our response: We have corrected the spelling mistake and replaced it with another word.
Reviewer 4 Report
The author has made a lot of descriptions about Consumer IoT Device, reviewed a lot of relevant literature, and provided theoretical support for practitioners. There are a few comments for the authors to further improve the paper's quality.
The author should make a final review to address all comments from reviewers and do a final editing and full revision of the language and correction of any typos.
IoT device vulnerabilities keep appearing in the paper, but it is confusing me, please explain in detail what it means in this paper?
The introduction lists a lot of literature, but does not directly describe the challenges of IoT device vulnerabilities, and it is suggested to rewrite the content of the introduction.
The pictures in the paper are not clear enough. It is recommended to use high-definition pictures.
How is the classifier in Figure 4 linked to the different module? Why is it designed this way? Please describe in detail the working mechanism of the proposed frame.
Some reference are missing, for example,
IIBE: An Improved Identity-Based Encryption Algorithm for WSN Security", Security and Communication Networks, vol. 2021, Article ID 8527068, 8 pages, 2021. https://doi.org/10.1155/2021/8527068
Ensemble unsupervised autoencoders and Gaussian mixture model for cyberattack detection. Information Processing & Management, 59(2), 102844.
Immune multipath reliable transmission with fault tolerance in wireless sensor networks. In Bio-inspired Computing–Theories and Applications: 11th International Conference, BIC-TA 2016, Xi'an, China, October 28-30, 2016, Revised Selected Papers, Part II 11 (pp. 513-517). Springer Singapore.
Author Response
We would like to thank you for your suggestions and comments. They are greatly appreciated.
Reviewer comment: The author has made a lot of descriptions about Consumer IoT Device, reviewed a lot of relevant literature, and provided theoretical support for practitioners. There are a few comments for the authors to further improve the paper's quality.
Our response: Thank you for your positive comments. They are greatly appreciated.
Reviewer comment: The author should make a final review to address all comments from reviewers and do a final editing and full revision of the language and correction of any typos.
Our response: We have carefully addressed all the comments.
Reviewer comment: IoT device vulnerabilities keep appearing in the paper, but it is confusing me, please explain in detail what it means in this paper?
Our response: We have defined IoT vulnerability in the second paragraph of the introduction section. We hope this addresses your concerns.
Reviewer comment: The introduction lists a lot of literature, but does not directly describe the challenges of IoT device vulnerabilities, and it is suggested to rewrite the content of the introduction.
Our response: We have rewritten the introduction as suggested. We have also added paragraph 3 that describes the challenges of IoT device vulnerabilities.
Reviewer comment: The pictures in the paper are not clear enough. It is recommended to use high-definition pictures.
Our response: Thank you for your comment. We have redrawn the figures as suggested.
Reviewer comment: How is the classifier in Figure 4 linked to the different module? Why is it designed this way? Please describe in detail the working mechanism of the proposed frame.
Our response: We think this comment is related to another paper. Our work is not about classifiers and different modules. Figure 4 in our paper does not have classifiers and modules.
Reviewer comment: Some reference are missing, for example,
- IIBE: An Improved Identity-Based Encryption Algorithm for WSN Security", Security and Communication Networks, vol. 2021, Article ID 8527068, 8 pages, 2021. https://doi.org/10.1155/2021/8527068
- Ensemble unsupervised autoencoders and Gaussian mixture model for cyberattack detection. Information Processing & Management, 59(2), 102844.
- Immune multipath reliable transmission with fault tolerance in wireless sensor networks. In Bio-inspired Computing–Theories and Applications: 11th International Conference, BIC-TA 2016, Xi'an, China, October 28-30, 2016, Revised Selected Papers, Part II 11 (pp. 513-517). Springer Singapore.
Our response: Our paper is on IoT vulnerability assessment frameworks. We thoroughly examined the three papers recommended by the reviewer in terms of their relevance to our paper. The first paper proposes an identity-based encryption algorithm (IIBE) that can effectively simplify the key generation process, reduce the network traffic, and improve the network security. The second paper proposes an ensemble framework of multichannel network anomaly detection model that combines deep autoencoders and the Gaussian mixture model (GMM). The third paper is about transmission reliability of wireless sensor networks and proposed an immune based multipath transmission algorithm for the case of fault of nodes or links affecting the stability and reliability of network. We have not been able to establish the link between these papers and the topic covered in our paper.
Round 2
Reviewer 1 Report
Thank you for making all the revisions. However, I still think the critical insight could be clearer. Additionally, and although the conclusion has been expanded, I think that more work should be done on the flow and language use within this section. Perhaps conclude briefly and discuss the future research opportunities into a standalone section above the conclusion.
Finally, I see the list of final review articles in the references column, but do not see the mapping table which usually accompanies a review. Can this be included in an Appendix? Table 4 is not quite a complete mapping table.
Author Response
Thank you very much for your suggestions. We hope we have attended to all your concerns in the updated version of the paper.
Reviewer: Adding critical insight
Response: We have done that.
Reviewer: Perhaps conclude briefly and discuss the future research opportunities into a standalone section above the conclusion.
Response: We have created a new section (section 6) called future direction. We have also reworked the conclusion section.
Reviewer: the mapping table which usually accompanies a review.
Response: We have included table 5 as requested.
Reviewer 2 Report
The structure of the narrative should be worked out more clearly.
There are difficulties with maintaining attention to the main idea.
Author Response
Thank you very much for your suggestions. We hope we have attended to all your concerns in the updated version of the paper.
Reviewer: The structure of the narrative should be worked out more clearly. There are difficulties with maintaining attention to the main idea.
Response: Thank you for your suggestion. We have reworked on the paper as you suggested.
