Intelligent Platform for Automating Vulnerability Detection in Web Applications

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

This paper proposes an intelligent platform that automates vulnerability detection in web applications using the ZAP scanning tool. The platform integrates Python-based REST APIs and ZAP’s automation framework for efficient scanning with minimal operator intervention. It claims to simplify vulnerability analysis by presenting results in a user-friendly format, focusing on OWASP Top 10 vulnerabilities. Experimental evaluations include tests on deliberately vulnerable applications and CMS platforms. The paper highlights the system’s ability to detect critical vulnerabilities but acknowledges limitations in authenticated scans. Overall, the system offers a low-cost, scalable, and accessible solution for web application security.

Detailed Comments

1. The introduction lacks a clear articulation of the novelty of this platform compared to existing solutions. While the focus on automation is commendable, the discussion does not sufficiently distinguish this system from related work using ZAP or similar tools. Clarify what unique contributions this system brings.

2. The evaluation primarily uses intentionally vulnerable applications and locally hosted CMS platforms, which may not reflect real-world conditions. The paper should test against more diverse, real-world applications to validate robustness under various operational environments.

3. Although qualitative results are presented, quantitative metrics such as detection accuracy, false positive/negative rates, and computational performance are missing. Adding these would significantly strengthen the evaluation.

4. The limitations in handling authenticated scans are discussed, but the technical reasons and potential remedies are unclear. Provide more technical details on the constraints faced and explore ways to address them.

5. The paper mentions integration with ZAP via a REST API and the use of YAML for automation plans, but the implementation specifics are superficial. Include details on API endpoints, system configurations, and any challenges encountered during integration.

6. While future work is briefly outlined, it lacks depth. Expanding on how additional tools (e.g., Arachni, Nikto) or advanced features like AI-driven anomaly detection will enhance the system could make the research direction more compelling.

Author Response

Comment 1 - The introduction lacks a clear articulation of the novelty of this platform compared to existing solutions. While the focus on automation is commendable, the discussion does not sufficiently distinguish this system from related work using ZAP or similar tools. Clarify what unique contributions this system brings.

Response 1 - Thank you for seeking clarifications. We added to the introduction a paragraph that identifies the contribution of the system that was developed. Moreover,  the results are presented in a secure, clear and organized way for the user. The main advantage of this solution is at the level of automation.

Comment 2 - The evaluation primarily uses intentionally vulnerable applications and locally hosted CMS platforms, which may not reflect real-world conditions. The paper should test against more diverse, real-world applications to validate robustness under various operational environments.

Response 2 - The system was also tested on a real website, the Fenix ISCTE platform. The website tested was the site in a quality environment, but it involved authentication and the system was able to identify the authentication method and perform an authenticated analysis. Information about this test has been added  in the Tests and Discussion of Results section.

Comment 3 - Although qualitative results are presented, quantitative metrics such as detection accuracy, false positive/negative rates, and computational performance are missing. Adding these would significantly strengthen the evaluation.

Response 3 - Thank you very much for your pertinent comment. The developed system focuses on automating the detection of vulnerabilities in web applications and does not validate their existence. In the tests section, the duration of each analysis has been added to the analysis, these times have been analyzed by target groups, the purposefully vulnerable applications and in the other group the CMS and the Fenix Iscte application. Regarding false positives, the system does not generate false positives, all alerts have a Confidence attribute which indicates the confidence with which that alert was generated, information on this topic has also been added to the Results sub-section within the Design and Development section.

Comment 4 - The limitations in handling authenticated scans are discussed, but the technical reasons and potential remedies are unclear. Provide more technical details on the constraints faced and explore ways to address them.

Response 4 - Thank you for your valuable comment, to address it we have added details about authentication process in the following text: “Additionally, Browser-based Authentication was implemented through the Automation Framework. The Browser-based Authentication in ZAP leverages Selenium, a browser automation tool, to handle login processes that require interaction with a graphical user interface. Through this method, ZAP launches a browser instance (either visible or headless) to simulate user actions such as filling in login credentials, clicking buttons, and interacting with dynamic elements like JavaScript-based forms or CAPTCHA solutions. Once the authentication is successful, ZAP captures session cookies or tokens from the browser, enabling automated authenticated scans of protected resources. This approach is particularly effective for complex login flows, such as Single Sign-On (SSO) or applications with heavy JavaScript frontends, although it has limitations, such as the inability to manage logins triggered exclusively by the "Enter" key or scenarios involving non-graphical APIs. By using this type of authentication, the system handles authentication intelligently, thus making it possible to automate this process.” Additionally, in conclusion we have proposed a potential approach for handling authentication failures, such as testing the application in a controlled environment, disabling the authentication component if possible.

Comment 5 - The paper mentions integration with ZAP via a REST API and the use of YAML for automation plans, but the implementation specifics are superficial. Include details on API endpoints, system configurations, and any challenges encountered during integration.

Response 5 - Thank you for raising this point and for your suggestion. In order to improve the paper, in the Design and Development section we have added information about the end points that were used in the solution we developed.

Comment 6 - While future work is briefly outlined, it lacks depth. Expanding on how additional tools (e.g., Arachni, Nikto) or advanced features like AI-driven anomaly detection will enhance the system could make the research direction more compelling.

Response 6 - Thank you for your thoughtful comment. We have revised the manuscript to provide a more detailed discussion of future work, including how advanced features could enhance the system's capabilities. We hope these changes address your concerns and clarify the research direction.

Reviewer 2 Report

Comments and Suggestions for Authors

The manuscript addresses a critical issue in cybersecurity by proposing an accessible, automated platform for vulnerability detection. The integration of open-source technologies aligns with contemporary needs for scalable and cost-efficient solutions. Authors need to do a major update and write a very clear scientific study

Comments for author File: Comments.pdf

Author Response

Comment 1 - What were the criteria for selecting ZAP as the primary scanning tool, and how do its detection capabilities compare to other similar tools in identifying vulnerabilities across diverse categories?

Response 1 - Thank you for seeking clarifications. In order to provide a good support we improved the paper by explaining the reasons why ZAP was chosen over the other tools that were mentioned in the related work section. Firstly, scientific studies and benchmarking articles consistently highlight ZAP's superior performance in detecting vulnerabilities in web applications compared to other scanners. ZAP offers advanced automation solutions, which are perfectly suited to the purpose of the work. ZAP has an API and automation frameworks that provide full control over the scanner without requiring interaction through the interface. Another critical factor was ZAP's active and supportive community, which proved invaluable in solving challenges. Finally, ZAP allows granular control of the scanner's results, making it possible to customize the results. This capability was crucial in adapting the system to provide clear and organized reports that met the specific needs of the users.

Comment 2 - What scientific principles support the decision to prioritize ZAP over other tools during the implementation phase, and can we extend the system's architecture to support multi-tool integration?

Response 2 - I've added explanations and the reasons why ZAP was chosen over the other tools to the related work chapter. First, scientific studies and benchmarking articles consistently highlight ZAP’s superior performance in detecting vulnerabilities in web applications compared to other scanners. Additionally, ZAP offers advanced automation solutions. Another critical factor was ZAP’s active and supportive community, which proved invaluable in addressing challenges, such as defining and implementing a complete and accurate automation plan. Finally, ZAP allows granular control over scan results, enabling the customization of outputs.

Comment 3 - How does the platform's approach to handling authentication scenarios align with existing methodologies, and what are the implications of its limitations on the scientific robustness of the vulnerability detection process?

Response 3 - Thank you for your valuable comment. We have added further details about the authentication process, including how it operates and also the mechanisms used to maintain session persistence. If the system can't detect the information needed to deal with authentication on a target, there are 2 options: disable authentication during the test and carry it out in a controlled environment (solution suggested in the article) or carry out the analysis without authentication and then we'll only get to the depth that a user without authentication would have, and this analysis will be much shallower and poorer.

Comment 4 - Could we introduce experimental design modifications to systematically evaluate the platform's performance across all OWASP Top 10 vulnerability categories, including those underrepresented in the current testing phase?

Response 4 - It's a very pertinent question. It might be something to consider in the future.

Reviewer 3 Report

Comments and Suggestions for Authors

The article is well-written overall and requires only minor changes.

However, the literature review is somewhat limited and mostly relies on sources that are quite outdated, with many references being over five years old. I suggest expanding and updating the references to include more recent studies.

Regarding reference 25, could you clarify what this refers to? I do not fully understand its relevance.

The sentence "highlights the most critical risks for web applications" should briefly discuss these risks in more detail.

On page 2, the phrase "This study..." should be moved to a new section that outlines the aims and objectives of the study.

Additionally, the introduction is quite brief and would benefit from being extended with more references and relevant case studies.

The "Related Work" section is not well-written and should be improved. It would be helpful to include a table summarizing the studies reviewed, specifically those related to Python-based vulnerability detection platforms and applications.

Section 3 is very well written—congratulations!

All figures should be referenced in the text and explained accordingly (e.g., Fig. 1).

On page 6, section 3.3.1, where is the actual framework and Python code that would allow us to replicate and test the application? Alternatively, providing the YAML or other property files would be useful.

Why are figures 2, 3, etc., presented as images rather than tables?

The AJCA Spider and Active Scan tools should be explained in more detail.

Figures 4, 5, and 6 are not particularly useful in their current form and could be revised or replaced.

In section 4:

• The phrase "running Ubuntu 20.04 LTS 352 (Focal Fossa)"—why was Windows not considered? Additionally, could this be done using low-power, low-cost mini-computers, such as Raspberry Pi or NVIDIA Jetson?

• "The system was further tested on..."—why was testing limited to PHP-based CMS only?

Section 4.1, Table 1 is excellent work and is quite interesting!

For Table 2, please provide further explanations of the results, particularly with respect to WordPress.

Lastly, the conclusions section should include suggestions for future work related to this research.

Author Response

Comment 1 - The article is well-written overall and requires only minor changes. However, the literature review is somewhat limited and mostly relies on sources that are quite outdated, with many references being over five years old. I suggest expanding and updating the references to include more recent studies.

Response 1 - Thank for your suggestions. We reviewed the references and we updated the list considering more recent studies.

Comment 2 - Regarding reference 25, could you clarify what this refers to? I do not fully understand its relevance.

Response 2 - Thank you for seeking clarifications. The reference 25 is related to the fact that the appropriate permissions are required before analyzing a website. This refers to a law prohibiting the exploitation of a website without permission.

Comment 3 - The sentence "highlights the most critical risks for web applications" should briefly discuss these risks in more detail.

Response 3 - Thank you for your suggestion. A paragraph has been added to the Related Work which identifies the top 10 web application risks according to the OWASP Top 10 ( as well as a brief summary of what they represent).

Comment 4 - On page 2, the phrase "This study..." should be moved to a new section that outlines the aims and objectives of the study.

Response 4 - Thank you very much for your comment. The introduction was changed in order to clarify the aim of the study.

Comment 5 - Additionally, the introduction is quite brief and would benefit from being extended with more references and relevant case studies.

Response 5 - Thank you for the reinforcement of your opinion. As we referred before, we reviewed the paper and updated our references. The changes are highlighted in the introduction.

Comment 6 - The "Related Work" section is not well-written and should be improved. It would be helpful to include a table summarizing the studies reviewed, specifically those related to Python-based vulnerability detection platforms and applications.

Response 6 - Thank you for the reinforcement of your opinion. We reviewed the paper and updated the related work section and also our references.

Comment 7 - Section 3 is very well written—congratulations!

Response 7 - Thank your for your kind comment. We appreciate your feedback and we developed efforts in order to improve the paper readability.

Comment 8 - All figures should be referenced in the text and explained accordingly (e.g., Fig. 1).

Response 8 - Thank you for raising this point. We reviewed the paper carefully in order to solve this kind of issues that you mentioned.

Comment 9 - On page 6, section 3.3.1, where is the actual framework and Python code that would allow us to replicate and test the application? Alternatively, providing the YAML or other property files would be useful.

Response 9 - Thank you for seeking clarification. We have mentioned the link to the github repository of the developed system in our references section. The repositories are public and by following the instructions in the README file, other users will be able to use the system.

Comment 10 - Why are figures 2, 3, etc., presented as images rather than tables?

Response 10 -Thank you for seeking clarifications. Figures 2 is an example of the target file that should has to be filled in by the user. Figure 3 represents  an example of an automation plan. These images serve to clarify what I am talking about in that section and make it easier for the reader to understand.

Comment 11 - The AJCA Spider and Active Scan tools should be explained in more detail.

Response 11 - Thank you for you suggestion. Ajax Spider and Active Scan are types of analysis that are present in the jobs section of the automation plan. This information can be found in the Design and Development section.

Comment 12 - Figures 4, 5, and 6 are not particularly useful in their current form and could be revised or replaced.

Response 12 - Thank you for your suggestion. We reviewed the figures, and we also performed the required measures in order to provide a good support for our text.

Comment 13 - In section 4:

Response 13 -

• The phrase "running Ubuntu 20.04 LS 352 (Focal Fossa)"—why was Windows not considered? Additionally, could this be done using low-power, low-cost mini-computers, such as Raspberry Pi or NVIDIA Jetson?

Thank you for seeking clarifications. The main idea is to offer an open-source (free) solution that only requires low hardware resources. Ubuntu is used because it's free, unlike Windows. The system was also designed to run on Raspberry Pi, this is possible because the system was developed taking into account the fact that it requires little computer resources to operate.

• "The system was further tested on..."—why was testing limited to PHP-based CMS only?

Thank you for raising this point. The system was also tested on a real website, the Fenix ISCTE platform. The website tested was in a quality environment, and the system was able to identify the authentication method and also perform an authenticated analysis. We provided more Information about this test in Tests and Discussion of Results section of the paper.

Comment 14 - For Table 2, please provide further explanations of the results, particularly with respect to WordPress.

Response 14 - Thank you for your comment. We have carefully reviewed the section concerning Table 2. The purpose of the results analysis is primarily to highlight the developed system's analytical capabilities rather than to provide a detailed assessment of the specific risks associated with the generated alerts. We believe this approach aligns with the study's primary objectives and scope.

Comment 15 - Lastly, the conclusions section should include suggestions for future work related to this research.

Response 15 - Thank you for your suggestions. We reviewed this section and we also improved our conclusions and future work.

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

The revision addressed my concerns. Thus I recommend the acceptance.

Author Response

Comment 1 - The revision addressed my concerns. Thus I recommend the acceptance.

Response 1 - We would like to thank the reviewer for all the relevant comments.

Reviewer 2 Report

Comments and Suggestions for Authors

The manuscript presents an automated framework for vulnerability discovery, utilizing ZAP for analysis via a Python-based architecture. It highlights low user involvement and suitability in resource-limited settings. The design incorporates elements such authentication management, flexible system components, and extensive reporting. Testing is performed on deliberately susceptible programs, content management system platforms, and an actual website. The outcomes are classified according to the OWASP Top 10 methodology. Limitations in certain vulnerability categories and the analysis duration for bigger systems are examined, along with recommendations for future enhancements.

The book offers a robust technical solution for automating vulnerability identification in online applications; nevertheless, substantial enhancements are necessary prior to publication. The authors must enhance the testing framework by using supplementary tools such as Arachni or Nikto to deliver a more thorough evaluation. The platform's failure to identify vulnerabilities in several OWASP categories, including Cryptographic Failures and Outdated Components, requires rectification to improve its resilience. Challenges associated with managing intricate authentication circumstances must be addressed or elucidated with viable solutions. The text would gain from a more robust scientific basis by contrasting the proposed platform with current tools and frameworks. Enhancements in presenting clarity, especially in elucidating testing techniques and offering more accessible visual aids, are essential. By resolving these concerns, the work can serve as a significant contribution to the discipline.

The manuscript presents a well-constructed technical solution that addresses a critical gap in cybersecurity for non-expert users. However, its scope and scientific rigor need substantial improvement before publication in a peer-reviewed journal.

Author Response

Comment 1 - The book offers a robust technical solution for automating vulnerability identification in online applications; nevertheless, substantial enhancements are necessary prior to publication.

Response 1 - Thank you for your comments. However we don’t understand to what book are you refering to.

Comment 2 - The authors must enhance the testing framework by using supplementary tools such as Arachni or Nikto to deliver a more thorough evaluation.

Response 2 - Thank you for this comment. This is certainly something that we would like to address on the future, and is in fact addressed by our article in the “Conclusions” section. “First, integrating additional vulnerability analysis tools, such as Nikto or Arachni, would enable a more comprehensive and precise analysis, broadening the range of vulnerabilities the system can detect”. However, we must again stress the fact that the development solution is independent of the type of vulnerability scanner used, since what it aims is to automate the web applications vulnerability scanning process, while using multiple scanners. For our implementation, we opt by using ZAP and all the article is based on the usage of this specific scanner.

Comment 3 - The platform's failure to identify vulnerabilities in several OWASP categories, including Cryptographic Failures and Outdated Components, requires rectification to improve its resilience.

Response 3 - Thank you for this comment. Once again, we would like to clarify that our work does not identify vulnerabilities. What it does is to provide an automated and scalable environment that integrates specific web application vulnerability scanners that identify vulnerabilities. Therefore, the failure to identify those vulnerabilities is due to the fact that the specific web application vulnerabilities that were detected were not in these specific categories. So, there is nothing that we can improve in our solution to cope with this comment. This is simply the results of the analysed applications and the scanners that were used.

Comment 4 - Challenges associated with managing intricate authentication circumstances must be addressed or elucidated with viable solutions.

Response 4 - Thank you for your comment. In the latest version of our article we have already provided an in-depth review that clarifies this issue.

Comment 5 - The text would gain from a more robust scientific basis by contrasting the proposed platform with current tools and frameworks. Enhancements in presenting clarity, especially in elucidating testing techniques and offering more accessible visual aids, are essential. By resolving these concerns, the work can serve as a significant contribution to the discipline.

Response 5 - Thank you for your comment. We believe that the latest version of our article already addresses these concerns by the reviewers. We have provide text with clarifications and additional visual aids to clarify the description of the system, and how it was tested and validated.

Comment 6 - The manuscript presents a well-constructed technical solution that addresses a critical gap in cybersecurity for non-expert users. However, its scope and scientific rigor need substantial improvement before publication in a peer-reviewed journal.

Response 6 - Thank you for your comment. We believe that the latest version of our article already addresses these concerns by the reviewers.