Review Reports

Round 1

Reviewer 1 Report (New Reviewer)

The manuscript is an extended version of the published paper online (https://www.researchgate.net/publication/360188915_Predicting_architectural_design_patterns_via_machine_learning_algorithms).

It deals with a machine learning-based approach for detecting architectural design patterns from source code from GitHub, and detects MVVM and MVP architectures based on 8 ML techniques. Research design and development are well-organized.

The contributions deliver direct knowledge in the field, but there are minor issues with the manuscript for revisions. 

- In Introduction, there is one subsection; I recommend removing the subsection title (not contents).

- Scientific representation is required. Some revisions are required for error corrections. I recommend consulting a manuscript editing service.

- No specific algorithms or pseudocodes are presented for presented methods.

- Some paragraphs are too long and some are too short. Reconstruction is required.

- Some tables can be converted to figures for visibility.

- Sections 6-7 can be merged.

- Some figures (e.g., Figure 2) are hard to recognize due to the small font size.

- The authors should justify why "Java" language is considered. Plus, detecting the design patterns (MVVM/MVP) is important topic in the field?

Author Response

Thank you a lot for your reviews, we have written a response letter which is included as a PDF file in the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report (New Reviewer)

The manuscript is well written. 

Authors must:

1] Add more recent and relevant references to show stronger proof of concept to their contributions.

2] Must add more visual results [graph etc..] to show how the model performed.

3] Add a bigger comparison table based on new references with some very strong existing research findings.

 

 

Author Response

Thank you a lot for your reviews, we have written a response letter which is included as a PDF file in the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report (New Reviewer)

The paper addresses an interesting and evolving research topic that is using ML to derive architectural design patterns. 

The main idea is to adopt different machine learning models and evaluate them according to some measures such accuracy, precision, recall, and F1-score  in order to define what code metrics contribute to enhance the prediction of architectural design patterns, specifically MVVM and MVP.

 

The analysis is performed on java projects retrieved from GitHub. 

Data are analysed  using eight popular machine learning models. 

 

The paper presents not a new ML approach or an enhancement of an existing ML approach but a comparative evaluation of existing ML approaches for predicting design patterns.

Which is the difference with respect to the work in [14] a part the smallest dataset in which the approach has been applied? The authors should better explain their contribution with respect to the work in [14].

More details about how the metrics are defined and derived need to be provided.

Which are examples of software architecture degradation or architectural inconsistencies found with the proposed approach?

The authors should provide more details about MVVM and MVP design patterns they say to be able to predict.

 

 

Minor

we propose a an approach --> we propose  an approach line 132

In this paper we present presents --> In this paper we present line 6

For this reason we we select-->For this reason  we select  line 242

Fig2 is too small

Author Response

Thank you a lot for your reviews, we have written a response letter which is included as a PDF file in the attachment.

Author Response File: Author Response.pdf

Reviewer 4 Report (Previous Reviewer 1)

I recommend a major revision based on the below points.  Please, add a point-to-point response to each comment in your revision:

-          I am not convinced about the novelty of the manuscript. 

-          The abstract is not technical and needs to highlight the research gap clearly.

-          The abstract also missed statistical information about the results.

-          The structure of the paper is vague. The paper needs to be restructured.

-          Don't add heading over heading. Add a few lines related to the detail of a particular section before starting a sub-section.

-          In the fig 1, have you performed any validation?

-          3.1 is the data extraction, whereas I cannot see that in the fig 1?

-          3.3 heading is machine learning Models, why the M is capital of models and not the L in learning?

-          The effect of table 1 parameters, is not fully discussed.

-          Proofread your paper from a native English speaker. There are many typos and grammar mistakes. 

-          At the end of the Introduction section, add the contributions clearly. See this paper for reference and citation ' A Novel Framework for Prognostic Factors Identification of Malignant Mesothelioma through Association Rule Mining. The mentioned contributions are not clear.

-          Related work/background/literature review should have a threat to a validity section. At the start of the background section, add a threat to a validity section. In that section, state the search strings and databases that have been explored to find the related work. See the below papers for references and citations 'Performance comparison and current challenges of using machine learning techniques in cybersecurity' and 'A Survey on Machine Learning Techniques for Cyber Security in the Last Decade'.

-          The literature needs to be sub-divided into multiple sub-sections.

-          Need to state the rationale behind the obtained results?

-          Add the below papers to your literature: Taxonomy of automated software testing tools, The Impact of Artificial Intelligence and Robotics on the Future Employment Opportunities, Security quality assurance through penetration testing,

-          Comparison with the state-of-the-art is missed. You need to compare your method with the ground truth. 

-          It needs to add the reasons why these metrics are used for comparison. 

-          Add the discussion related to the time complexity factor of AI models. See this paper for reference and citation 'Cyber Threat Detection Using Machine Learning Techniques: A Performance Evaluation Perspective'.

Overall, the paper has many inconsistencies, and the contributions are not clear. The results are not compared with the ground truth properly. Limitations are not provided in their current approach. Future directions are not clearly stated.

I am looking forward to seeing your revised version. 

 

All the best. 

Author Response

Thank you a lot for your reviews, we have written a response letter which is included as a PDF file in the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 3 Report (New Reviewer)

The authors have implemented most of my previous comments.

Reviewer 4 Report (Previous Reviewer 1)

The authors have addressed my comments sufficiently. 

 

Congratulations. 

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.

Round 1

Reviewer 1 Report

I recommend a major revision based on the below points.  Please, add a point to point response to each comment in your revision:

-          I am not convinced about the novelty of the manuscript. 

-          The abstract is not technical and needs to clearly highlight the research gap.

-          The abstract also missed statistical information about the results.

-          The structure of the paper is vague. The paper needs to be restructured.

-          Don't add heading over heading. Add a few lines related to the detail of a particular section before starting a sub-section.

-          Proofread your paper from a native English speaker. There are many typos and grammar mistakes. 

-          At the end of the Introduction section, add the contributions clearly. See this paper for reference and citation ' A Novel Framework for Prognostic Factors Identification of Malignant Mesothelioma through Association Rule Mining.

-          Related work/background/literature review should have a threat to a validity section. At the start of the background section, add a threat to a validity section. In that section, state the search strings and databases that have been explored to find the related work. See the below papers for references and citations 'Performance comparison and current challenges of using machine learning techniques in cybersecurity' and 'A Survey on Machine Learning Techniques for Cyber Security in the Last Decade'.

-          The literature needs to be sub-divided into multiple sub-sections.

-          Add the below papers to your literature: Taxonomy of automated software testing tools, Security quality assurance through penetration testing, A comparative study of numerical analysis packages

-          Add the statistical test (t-test/p-test/ANOVA, whichever is applicable) to compare your method with others. 

-          Comparison with the state-of-the-art is missed. You need to compare your method with the ground truth. 

-          You need to add a separate section for limitations of your approach and Future directions. 

-          It needs to add the reasons why these metrics are used for comparison. 

-          Add the discussion related to the time complexity factor of AI models. See this paper for reference and citation 'Cyber Threat Detection Using Machine Learning Techniques: A Performance Evaluation Perspective'.

Overall, the paper has many inconsistencies, and the contributions are not clear. The results are not compared with the ground truth properly. Limitations are not provided in their current approach. Future directions are not clearly stated.

I am looking forward to seeing your revised version. 

 

All the best. 

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

The article is devoted to developing an approach based on machine learning to determine the software architecture. A systematic approach to software architecture is well defined in academia as software architecture patterns, which are well practiced in industry. The study's relevance is justified by the fact that the software architecture is configured depending on the context of the project and functional and non-functional requirements. This process is error-prone because it requires human participation to consider aspects that affect the architecture. Machine learning methods are successfully applied in software development. Understanding the relationship between source code and other program object entities is essential. Through this understanding, we can understand the various aspects that affect software architecture. Therefore, this article presents a machine learning-based approach to software architecture determination. The authors present a labeled dataset consisting of 5973 data points from GitHub. The authors then apply nine machine learning techniques to determine software architecture from source code metrics. The results show that the software architecture can be predicted from the source code since the performance of the machine learning models is satisfactory.

Some shortcomings need to be corrected.

1. The methodology section should be expanded with a detailed description of each pipeline step.
2. The selection of eight machine learning models should be justified. Why do authors select such approaches? That selection should be grounded on current research analysis.
3. The dividing of training and testing samples proposed by the authors should be grounded.
4. While authors use well-known approaches, one of them, Neural Network, is not standardized. The architecture of the proposed neural network should be described in detail.
5. The discussion section should be included comparing obtained results with other research in the field.
6. The conclusion section should include numerical results.
7. While the authors use standard models from sklearn and interpret libraries, the contribution of the research should be described in detail.
8. The scientific and practical novelty should be highlighted.
9. Self-citations 40 and 41 do not carry any semantic meaning within the study.

In summarizing my comments, I recommend that the manuscript is accepted after major revision. 

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Congratulations

Author Response

We would like to thank you for your valuable reviews. We are attaching our response letter to your feedbacks.

Author Response File: Author Response.pdf

Reviewer 2 Report

Authors have updated an article considering some of the comments, but most of reviewer's comments and recommendations are still needed to be corrected:

1. The methodology section should be expanded with a detailed description of each pipeline step.
2. The selection of eight machine learning models should be justified. Why do authors select such approaches? That selection should be grounded on current research analysis.
3. While authors use well-known approaches, one of them, Neural Network, is not standardized. The architecture of the proposed neural network should be described in detail.
4. The discussion section should be included comparing obtained results with other research in the field.
5. While the authors use standard models from sklearn and interpret libraries, the contribution of the research should be described in detail.
6. The scientific and practical novelty should be highlighted.
7. Authors have deleted self-citation from the reference list, but added some references that don't have any connections with the research.

In summarizing my comments, I recommend that the manuscript should have one more round of major revision. 

Author Response

We would like to thank you for your valuable reviews. We are attaching our response letter to your feedbacks.

Author Response File: Author Response.pdf

Round 3

Reviewer 2 Report

Authors have made some corrections according to reviewer's comments and recommendations. Since the authors use standard widely known methods, their contribution to the study is not clear.