Chidroid: A Mobile Android Application for Log Collection and Security Analysis in Healthcare and IoMT

Round 1

Reviewer 1 Report

The manuscript tackles an interesting subject and the (implementation-focused) work is presented in detail.

Nevertheless, some improvements can be made to the manuscript to further improve its quality. Such recommendations are provided below:

“as mentioned by the ENISA report”  -> as mentioned by ENISA’s Threat Landscape Report 2021 (ETL 2021)

The intro is a bit verbose in some parts (e.g., detailing how Ryuk works, when this is not the focus of the paper).

In section 1.2 the authors should clarify their definition of “edge”, where the log processing takes place. In many cases the Android devices themselves are considered edge devices. In fact, it is only until the reader reaches page 9 (Figure 2) that the architecture the authors have in mind is presented.

In the Methodology (Section 2) it would help to specify the actual devices that were used for testing (testbed). Furthermore, it would help to present the architecture of the tool (Section 3) before mentioning the underlying tools (like Logcat). These are anyway mentioned within Section 3 so the text could be merged there. They are not part of the methodology, nor can the reader put them in context at that stage in the paper.

In Section 4 it would help to also give at least a basic assessment of the performance impact that Chidroid introduces.

Finally, an important omission is the consideration for privacy concerns of the individuals using the devices monitored by Chidroid. Any relevant assumptions (e.g., that the edge devices doing the processing are somehow trusted, with access controls, etc.) should be explicitly stated, also considering future applications that the authors mention (e.g., creating a data pool of logs for ML applications would definitely necessitate the sanitisation/anonymisation of these logs). 

Author Response

Response to Reviewer 1.

Dear Reviewer,

Thank you for taking the time to review our research paper. We appreciate the effort you put into carefully reading and providing constructive feedback. Please note that the proofreading process has been completed and all of your suggestions have been applied to the manuscript. We believe that the revisions have significantly improved the overall quality and clarity of the paper. In response to your comments, we have made the following changes to our paper:

Comment 1: “as mentioned by the ENISA report”  -> as mentioned by ENISA’s Threat Landscape Report 2021 (ETL 2021)

Response 1: Proofreading process has been completed and all of your suggestions have been applied to the manuscript.

Comment 2: The intro is a bit verbose in some parts (e.g., detailing how Ryuk works, when this is not the focus of the paper).

Response 2: Thank you for the comment. Indeed, in the previous version of the paper, there was irrelevant text in some parts of the paper and would not fit in the introduction. In response to your suggestions, we have revised the Section 1 - Introduction to provide a clearer and more concise overview of the background and motivation for our study. We believe that the revised Introduction now effectively sets the stage for the rest of the paper and provides a strong foundation for our research.

Comment 3: “In section 1.2 the authors should clarify their definition of “edge”, where the log processing takes place. In many cases the Android devices themselves are considered edge devices. In fact, it is only until the reader reaches page 9 (Figure 2) that the architecture the authors have in mind is presented.”

Response 3: In response to your comment regarding the introduction of edge computing, we have added additional text throughout Section 1 – Introduction and clarified better in 1.2 – Contribution. Additional text was added throughout the research paper to provide a clearer and more comprehensive introduction to edge computing. We believe these additions will help increase the reader's understanding of edge computing and its relation to our research.

Comment 4: “In the Methodology (Section 2) it would help to specify the actual devices that were used for testing (testbed). Furthermore, it would help to present the architecture of the tool (Section 3) before mentioning the underlying tools (like Logcat). These are anyway mentioned within Section 3 so the text could be merged there. They are not part of the methodology, nor can the reader put them in context at that stage in the paper.”

Response 4: The reviewer is correct. We acknowledge that the testbed used in our research was described as a generic digital environment because the actual smart and IoMT devices are confidential due to licence agreements. In the new version of the paper, we clarify this to avoid confusion. We understand the importance of exploring the application of our system in real-world scenarios. We have added a brief description in section 5.1 – Future Work to present future work that includes a testbed for eHealth.

Regarding Section 3, the information flow was not clear in the previous version. In response to your comments regarding the structure of the paper, we have made some changes to improve the readability and flow of the paper. Specifically, we have merged information from Section 2 into Section 3 to provide a more linear introduction to the architecture and building blocks of Chidroid. This change will allow readers to gain a better understanding of our system and its components without having to jump between sections.

Comment 5: “In Section 4 it would help to also give at least a basic assessment of the performance impact that Chidroid introduces.

Response 5: In response to your comments regarding performance impact, we have added a basic assessment of the performance impact in Section 3.9 of the paper. In the same section was added an evaluation of the CPU usage and RAM consumption, which provides a comprehensive overview of the performance impact of our system. The section on Methodology (Section 2) was updated as well. Moreover, in response to your suggestions, we have added an assessment and further ways of assessing Chidroid in terms of performance impact as future work in Section 5.1. This will provide a comprehensive evaluation of the system's impact on resource utilization and performance, which will be useful for determining its potential as a solution for edge computing applications.

Comment 6: “Finally, an important omission is the consideration for privacy concerns of the individuals using the devices monitored by Chidroid. Any relevant assumptions (e.g., that the edge devices doing the processing are somehow trusted, with access controls, etc.) should be explicitly stated, also considering future applications that the authors mention (e.g., creating a data pool of logs for ML applications would definitely necessitate the sanitisation/anonymisation of these logs).”

Response 6: Inded privacy concerns are of utmost importance. In response to your comment regarding security and privacy, we have added additional text to the methodology in Section 2 and Section 5 (Conclusion) to mention that privacy issues are significant and Chidroid includes privacy rules defined using the rules.toml file (Section 3 – Chidroid Architecture). However, we would like to mention that a complete privacy-preserving solution is out of the scope of the paper as the focus of this work is on log collection and distribution.

Once again, we thank you for your time and support. We appreciate your efforts in reviewing our work and providing us with constructive criticism.

Reviewer 2 Report

The authors proposed a mechanism to create datasets and then annalized by machine learning approaches to get to know the activities that took place on the devices. I think the work is good and supported by demonstrated experiments. I like ti the proposed work and only recommened proofreading and correcting minor typos.

Author Response

Dear Reviewer,

We appreciate your time and efforts in reviewing our research paper. We have considered your feedback and made the necessary minor corrections and proofreading. Thank you for helping us improve the quality of our work.

Reviewer 3 Report

The author has done a good job by creating a policy-based tool to examine and analyse security issues, However if the following suggestions are included, the article would be greatly improved.

1. I suggest the authors to with tabular references with characteristics of the previous proposed methods so that it is easy for the readers to follow. I have suggested few healthcare security related contemporary papers for author's reference to cite.

https://doi.org/10.1155/2022/8457116

https://doi.org/10.1155/2022/4167700. 

2. Usually, the Android App Store employs dynamically generated HTML pages so that the HTML texts displayed in the browser do not convey useful information, which is dynamically loaded from an underlying database. In that case, there was no detailed explanation of how authors retrieved app information. 

3.  https://doi.org/10.1155/2022/2500377 - In this work, the authors have used an authentication mechanism for healthcare mobile applications. How do you compare your work with this work?

4. Authors have generalised the work as 'security analysis' but I couldn’t find out any analysis made on the potential list of attacks in their log analysis.

5. Apart from the technical content, the paper should interest the non-technical readers / new comers as well. Basic questions like:

What are the prime issues related to data privacy and security are involved in using health apps?

What are the different security measures that can be made to overcome that?

6. In policies, the authors can consider HIPAA regulations on health care data perseverance in digital applications.

7. Though the work majorly deals with android, nowhere it is mentioned how it favours the chances against ios. Authors should mention a few points regarding that, at least in the future work.

8. Author is advised to follow the IMRAD format for the entire paper.

Author Response

Dear Reviewer,

We extend our gratitude for your thorough review of our research paper. Your feedback has been incredibly valuable and has helped us improve the quality of our work. In response to your comments, we have made the following changes:

Comment 1: “I suggest the authors to with tabular references with characteristics of the previous proposed methods so that it is easy for the readers to follow. I have suggested few healthcare security related contemporary papers for author's reference to cite. https://doi.org/10.1155/2022/8457116 https://doi.org/10.1155/2022/4167700.”

Response 1: Tabular references have been added for easier source location, and significant additions have been made to the related work (including one of the recommendations and six more) in section (Section 1.1) to provide a more comprehensive overview of the current state of research in our field.

Comment 2: “Usually, the Android App Store employs dynamically generated HTML pages so that the HTML texts displayed in the browser do not convey useful information, which is dynamically loaded from an underlying database. In that case, there was no detailed explanation of how authors retrieved app information. Answer directly.”

Response 2: Thank you for the comment. In the new version of the paper in section 2 – Methodology (paragraph 4), we mention that Chidroid retrieves the logs using Package manager (pm command), and Logcat, which provides a dynamic view of the log messages and allows for real-time analysis of the logs, rather than the XML pages or other files stored inside the device. With this technique, it is possible to track the changes in software package permissions.

Comment 3: “https://doi.org/10.1155/2022/2500377 - In this work, the authors have used an authentication mechanism for healthcare mobile applications. How do you compare your work with this work?”

Response 3: Thank you for the proposed article. We have included this paper as a reference as the authentication mechanism of that work can be integrated in Childroid for stronger security guarantees.

Comment 4: “Authors have generalised the work as 'security analysis' but I couldn’t find out any analysis made on the potential list of attacks in their log analysis.”

Response 4: The security analysis has been described further in Section 4, and a brief summary has been added to the conclusions and future work sections to provide a better overview of the security measures taken and highlight the importance of security in our research.

Comment 5: “Apart from the technical content, the paper should interest the non-technical readers / new comers as well. Basic questions like:  What are the prime issues related to data privacy and security are involved in using health apps? What are the different security measures that can be made to overcome that?”

Response 5: Indeed, we recognize that the prime issues related to data privacy and security require further investigation. To this end, in the new version of the paper, we have extended the introduction to cover prime issues related to data privacy and security in using heal apps. Additionally, we have briefly discussed security measures to overcome them. The context is in section 1 – Introduction and Section 5 - Conclusions. Moreover, new text has been added in section 2, Related work, to address this comment.

Comment 6: “In policies, the authors can consider HIPAA regulations on health care data perseverance in digital applications.”

Response 6: Indeed, HIPAA regulations are relevant to the scope of our paper. To this end, in the new version of the paper in section 3.5, we have included a brief analysis of HIPAA compliance issues to emphasize the importance of adhering to the standards for protecting sensitive medical data.

Comment 7: “Though the work majorly deals with android, nowhere it is mentioned how it favors the chances against iOS. Authors should mention a few points regarding that, at least in the future work.”

Response 7: While there is potential for extending the Chidroid system to support iOS devices, the project's current focus will remain solely on Android devices. The reason for this is that, while iOS is a widely used platform, Android's openness provides a greater opportunity for researching and addressing security and privacy challenges in a more comprehensive manner. This was added as future work in Section 5.1.

Comment 8: “Author is advised to follow the IMRAD format for the entire paper.”

Response 8: As suggested by another reviewer, reconstruction has been made to increase readability and follow the IMRAD format as best as possible.

Once again, we thank you for your time and support. We appreciate your efforts in reviewing our work and providing us with constructive criticism.