Deconstructing Risk Factors for Predicting Risk Assessment in Supply Chains Using Machine Learning
Round 1
Reviewer 1 Report
Dear Author(s)
Thank you for the opportunity to read your study, which is an attempt to propose a new model to automate and objectify the risk assessment in the supply chain using the machine learning algorithm.
The proposed topic is very interesting and the time perspective adopted by the authors allows them to capture the changes in the global risk landscape.
However, in its current stage, the manuscript can still be improved to be more valuable to the readers. Therefore, I encourage you to make some changes that I believe will be made this paper stronger. Please see my detailed comments below. Good luck with your research!
I have provided my comments as follows:
Title –I found it interesting, informative and encouraging to read.
Abstract – The key elements are included. I suggest a clear indication that a new risk assessment model is being proposed, not a new risk management methodology (v. 23-25).
Introduction - In my opinion, the introduction gives the reader an overview of the subject.
Literature review - I consider the literature review to be up-to-date, presented in a critical way. In my opinion, it provides a good basis for the self-study presented below.
Materials and Methods
I suggest that the authors consider changing the title of Fig. 1. In my view, it presents a risk assessment process rather than an overall risk management process.
What does "level of integrity" mean? (v. 292)
I encourage you to complete the work with all sub-parameters. In my opinion, the methodological description is too general and currently does not allow the study to be repeated by other researchers. I encourage you to give a more detailed description of the research process. In my opinion, the article would be stronger if the authors improved this part of the article.
Results
The results of the tests are presented correctly but not exhaustively. I recommend moving some of the information about the procedure to the 'materials and method' section.
Findings and discussion
Only findings are discussed. There is no discussion of the results obtained in the context of previous studies. In my opinion, the article would be stronger by supplementing this information.
Conclusions
The conclusions include the most important findings of the presented research. I also encourage you to consider supplementing the paper with information on the limitation of the study.
It was a pleasure to read this manuscript. There is a lot of potential in this paper to make an interesting contrubution to the risk management research. Overall, I find the idea of the study very valuable. I hope you find the above observations useful as you continue to further develop your study. Good luck with your paper!
Author Response
Thank you for the opportunity to read your study, which is an attempt to propose a new model to automate and objectify the risk assessment in the supply chain using the machine learning algorithm.
The proposed topic is very interesting and the time perspective adopted by the authors allows them to capture the changes in the global risk landscape.
However, in its current stage, the manuscript can still be improved to be more valuable to the readers. Therefore, I encourage you to make some changes that I believe will be made this paper stronger. Please see my detailed comments below. Good luck with your research!
I have provided my comments as follows:
Title –I found it interesting, informative and encouraging to read.
Abstract – The key elements are included. I suggest a clear indication that a new risk assessment model is being proposed, not a new risk management methodology (v. 23-25).
>>> per your comment we now clarified it in line 25. We also rewrote the entire abstract for additional clarifications.
Introduction - In my opinion, the introduction gives the reader an overview of the subject.
Literature review - I consider the literature review to be up-to-date, presented in a critical way. In my opinion, it provides a good basis for the self-study presented below.
Materials and Methods
I suggest that the authors consider changing the title of Fig. 1. In my view, it presents a risk assessment process rather than an overall risk management process.
>>> We have now changed the title of Fig. 1 as suggested
What does "level of integrity" mean? (v. 292)
>>> We clarified it in line 293
I encourage you to complete the work with all sub-parameters. In my opinion, the methodological description is too general and currently does not allow the study to be repeated by other researchers. I encourage you to give a more detailed description of the research process. In my opinion, the article would be stronger if the authors improved this part of the article.
>>> Per your comment we have now added a more detailed description of additional parameters on lines 318-381
Results
The results of the tests are presented correctly but not exhaustively. I recommend moving some of the information about the procedure to the 'materials and method' section.
>>> As suggested above, we added more examples to the ‘materials and methods’ section so we believe that the result section now is more informative and easier to follow.
Findings and discussion
Only findings are discussed. There is no discussion of the results obtained in the context of previous studies. In my opinion, the article would be stronger by supplementing this information.
>>> Thank you for that comment. Our model is novel and a stand-alone method for risk assessment but indeed parts of it could be utilized in the context of previous works. We addressed it in a new paragraph in Section 5.
Conclusions
The conclusions include the most important findings of the presented research. I also encourage you to consider supplementing the paper with information on the limitation of the study.
>>> We have now added a new paragraph in the conclusion that describes the foreseeable limitations of our work, and some future ideas of dealing with them.
It was a pleasure to read this manuscript. There is a lot of potential in this paper to make an interesting contrubution to the risk management research. Overall, I find the idea of the study very valuable. I hope you find the above observations useful as you continue to further develop your study. Good luck with your paper!
>>> Thank you so much for your review.
Reviewer 2 Report
The article presents a model using machine learning as a decision support tool for risk management assessment and control.
The article offers an interesting perspective validated by data collected from numerous companies subjected to performance audit.
Even if not detailed, the auditors' perspective may be very interesting to debrief and if possible, the article may include a brief section on this topic.
Some improvements may be added, as follows:
Check if all abbreviations are detailed.
Line 188 manage
Line 114 current
Fig 1 refers to fig. 1 instead of fig 2
Section 3.2.1 and line 428 define more in detail the domain / sector / industry.
Line 345 preliminary
Line 348 if possible, detail a bit more how the calculus have been made.
Good continuation !
Author Response
The article presents a model using machine learning as a decision support tool for risk management assessment and control.
The article offers an interesting perspective validated by data collected from numerous companies subjected to performance audit.
Even if not detailed, the auditors' perspective may be very interesting to debrief and if possible, the article may include a brief section on this topic.
>>> The auditor’s perspective is indeed interesting. However, we didn’t conduct interviews with them to get their perspective on the new model. We added this idea in the future work section, see last paragraph of the manuscript.
Some improvements may be added, as follows:
Check if all abbreviations are detailed.
Line 188 manage
Line 114 current
Fig 1 refers to fig. 1 instead of fig 2
>>> All the above now checked and corrected.
Section 3.2.1 and line 428 define more in detail the domain / sector / industry.
>>> We clarified this in lines 486-487
Line 345 preliminary
>>> Corrected
Line 348 if possible, detail a bit more how the calculus have been made.
>>> We now provided more details around lines 415-420
Good continuation !
Reviewer 3 Report
Title
Risk management in supply chains is mentioned in the abstract and keywords. Would it not be more appropriate to include this in the title of the article?
Abstract
It is quite long, especially the first paragraph. The first paragraph could be shortened and the second paragraph could add the contributions of the article. It lacks a clear aim of the paper and purpose, research implications, originaliy/value of paper.
Literature review
This part is not clear. I recommend dividing it by a few headings into a clearer structure to make it clear what this section is about. For example, the section focuses on Industry 4.0.
Very short, simple, no explanation of the different factors. If the authors state that they have reviewed the articles and selected the main factors accordingly, then it would also be useful to describe them in more detail (giving each factor at least a paragraph of explanation. Are all factors really relevant to the supply chain?
The question is whether the concept of supply chain should also be defined and how the authors understand it (how authors solve relationships between / among companies in supply chain network)?
Methodology
The authors report that they used data from 60 companies to apply the neural network method. Isn't that too few for the method? According to the literature, what is the minimum for the method? I recommend that this be included in the shortcomings of the paper.
There is a lack of further characterization of the data. Out of 60 enterprises, 10 may be large, 20 medium and 30 small enterprises and this may be misleading. It may be similar with the business sector. More characterisation of the data is needed.
Table 1 shows the different factors. The table then shows how one of the factors was constructed. In my opinion, the way each factor is constructed should be given for each factor.
The authors mention in the introduction as relevant risk factors in supply chains COVID-19, war in ukraine etc. But selected factors are rather related to internal environment of companies (e.g. corporate culture, payment control etc.). Perhaps this should be highlighted in the methodology.
In the paper, the authors discuss risk in the supply chain a lot. Supply chains and supply networks both describe the flow and movement of materials and information, by linking organisations together to serve the end-customer. Network describes a more complex structure, where organisations can be cross-linked and there are two-way exchanges between them (Harland et al., 2001). How do your selected factors represent this networked characteristic of supply chains? By this I mean, the relationships between individual firms?
Discussion
Does not include comparisons with other authors and studies. It needs to be expanded to have a scientific basis.
Conclusion
"As stated in the Introduction" - this is an unnecessary introductory sentence.
The article lacks practical and theoretical contributions.
The article does not state the shortcomings of the research.
Overall this article needs a major revision. In particular the addition of a literature review, methodology. Also, expanding the discussion and adding a conclusion. The title of the article needs to be considered, especially the supply chain context.
Comments for author File: 
Comments.pdf
Author Response
Title
Risk management in supply chains is mentioned in the abstract and keywords. Would it not be more appropriate to include this in the title of the article?
>>> We have now rephrased the Title based on your suggestion.
Abstract
It is quite long, especially the first paragraph. The first paragraph could be shortened and the second paragraph could add the contributions of the article. It lacks a clear aim of the paper and purpose, research implications, originaliy/value of paper.
>>> Thank you for this comment. We rewrote the abstract from scratch and integrated your suggestions.
Literature review
This part is not clear. I recommend dividing it by a few headings into a clearer structure to make it clear what this section is about. For example, the section focuses on Industry 4.0.
>>> We have now divided it according to your suggestion.
Very short, simple, no explanation of the different factors. If the authors state that they have reviewed the articles and selected the main factors accordingly, then it would also be useful to describe them in more detail (giving each factor at least a paragraph of explanation. Are all factors really relevant to the supply chain?
>>> We have now added new text to address this issue on the last paragraph.
The question is whether the concept of supply chain should also be defined and how the authors understand it (how authors solve relationships between / among companies in supply chain network)?
>>> We elaborated in lines 197-202
Methodology
The authors report that they used data from 60 companies to apply the neural network method. Isn't that too few for the method? According to the literature, what is the minimum for the method? I recommend that this be included in the shortcomings of the paper.
>>> The reviewer presents a very good point. Data on 60 companies is very impressive data set yet might be small for running ML algorithms. We did extensive work to optimize our algorithm so it will be able to deal with the size of the data, but indeed this is some basic limitation of the model. We hope that with more information in the future, we will attain a better accuracy results. We added text, as suggested by the reviewer, to address it in the new limitation paragraph on page 13-14.
There is a lack of further characterization of the data. Out of 60 enterprises, 10 may be large, 20 medium and 30 small enterprises and this may be misleading. It may be similar with the business sector. More characterisation of the data is needed.
>>> All our 60 selected organizations are of medium size (between 100 – 600 employees). We added new text on that around lines 499-501.
Table 1 shows the different factors. The table then shows how one of the factors was constructed. In my opinion, the way each factor is constructed should be given for each factor.
>>> As suggested, we now added more information of the specific factors in lines 354-393
The authors mention in the introduction as relevant risk factors in supply chains COVID-19, war in ukraine etc. But selected factors are rather related to internal environment of companies (e.g. corporate culture, payment control etc.). Perhaps this should be highlighted in the methodology.
>>> Per this comment, we have now highlighted the focus on internal factors in lines 197-202
In the paper, the authors discuss risk in the supply chain a lot. Supply chains and supply networks both describe the flow and movement of materials and information, by linking organisations together to serve the end-customer. Network describes a more complex structure, where organisations can be cross-linked and there are two-way exchanges between them (Harland et al., 2001). How do your selected factors represent this networked characteristic of supply chains? By this I mean, the relationships between individual firms?
>>> You raise a very interesting and important point. We address it when we audit the supply side and the risks relevant to sole supplier and inflow of raw materials in few sub-factors. Yet this model is a general methodology that can be modulated as required by specific needs of industry. The factors we selected are basic and, in the future, we will see the evolution.
Discussion
Does not include comparisons with other authors and studies. It needs to be expanded to have a scientific basis.
>>> This is an important comment and we've now corrected the paper by writing three new paragraphs in the Discussion section to address the work in the context of current literature. However, strict comparisons are not available as there are no similar work to compare against.
Conclusion
"As stated in the Introduction" - this is an unnecessary introductory sentence.
>>> Corrected
The article lacks practical and theoretical contributions.
>>> Corrected.
The article does not state the shortcomings of the research.
>>> Added a new paragraph to address limitations.
Overall this article needs a major revision. In particular the addition of a literature review, methodology. Also, expanding the discussion and adding a conclusion. The title of the article needs to be considered, especially the supply chain context.
Reviewer 4 Report
Comments:
This paper proposes a new methodology for risk assessment prediction using an Artificial Neural Network the idea is interesting. The authors have done a great job. But, I have a few censures regarding this submission. The major issues in the paper are in its current form. See the comments that follow:
In Abstract: lines 12-14, “Risk management in industry and along the supply chain has become more significant in recent years due to the increased complexity of the relationships between different components in the chain.” what industry? And which supply chain? Do you think the same supply chain applies to every industry (For example automobile and health care)? Please choose a specific industry, because risk dimensions for every sector are not the same in my opinion.
Our predictive model was constructed on unique real-world data of genuine risk assessment reports of over 60 factories and organizations in Israel.
In the introduction section: The authors need to clarify the relevance of the topic and why it is worth of investigation. Having a gap does not mean that the topic is worth investigating. In particular, being focused on a specific industry, I suggest adding some examples from literature explaining why the topic is relevant.
Literature review section: It is the main strength of this paper. However, if possible, it is suggested that try to a short and clear explanation of the problem description. The readers could understand the problem description easily.
In the materials and methods section: this is one of the major issues in this paper Identification and finalizing the risk factors, how can the authors conclude the risks? Why do not use any research method to finalize the risk factors? Example: Fuzzy Delphi method. That is to be without any literature support or industry experts’ input. It seems those risk factors were finalized purely based on the author's opinion.
In conclusion: I am recommended that this section be modified as follows, the discussion for this study appears to be too brief. What does the result imply? How is this consistent with other studies? The discussion should have two ways, “theoretical implications” and “practical implications.” The conclusion, limitations, and future research directions should follow the discussion section. Here, the others should highlight the study’s limitations based on the methodological design of the study. Authors should consider the generalizability of the result. Following this, future research directions can be proposed to address these limitations.
The authors need to take or cite more literature support to explain their point of view, if possible, try to add recently published work.
Finally, the language and style need to be deeply revised. I’m not a native speaker, but I found a few unclear sentences. Especially the introduction and numerical analysis (sections 3 & 4).
I wish the authors the best with this research
Comments for author File: 
Comments.docx
Author Response
This paper proposes a new methodology for risk assessment prediction using an Artificial Neural Network the idea is interesting. The authors have done a great job. But, I have a few censures regarding this submission. The major issues in the paper are in its current form. See the comments that follow:
In Abstract: lines 12-14, “Risk management in industry and along the supply chain has become more significant in recent years due to the increased complexity of the relationships between different components in the chain.” what industry? And which supply chain? Do you think the same supply chain applies to every industry (For example automobile and health care)? Please choose a specific industry, because risk dimensions for every sector are not the same in my opinion.
>>> We addressed this issue by rewriting the abstract to make it clearer. However, please note that our work’s aim is to propose a new concept and framework and not to be an accultured solution to a specific sector, rather to be a generic methodology to be evolved. This can also be seen as our real-world data set includes reports of over 60 different types of factories and organizations.
In the introduction section: The authors need to clarify the relevance of the topic and why it is worth of investigation. Having a gap does not mean that the topic is worth investigating. In particular, being focused on a specific industry, I suggest adding some examples from literature explaining why the topic is relevant.
>>> Thank you for that insightful comment. We have now added new text in the introduction (around lines 69-77) to address the motivation for filling the gap.
Literature review section: It is the main strength of this paper. However, if possible, it is suggested that try to a short and clear explanation of the problem description. The readers could understand the problem description easily.
>>> We have now divided the text into segments and cleared the overall section.
In the materials and methods section: this is one of the major issues in this paper Identification and finalizing the risk factors, how can the authors conclude the risks? Why do not use any research method to finalize the risk factors? Example: Fuzzy Delphi method. That is to be without any literature support or industry experts’ input. It seems those risk factors were finalized purely based on the author's opinion.
>>> In the literature there is no specific method that represent selection of risk factors. In our research the main focus is the framework and method of deconstruction to reduce human bias rather the specific best selection of risk factors. In our opinion the risk factors selection should be different to any industry or even a company based on its characteristics. We have added a new text in the Discussion section to address it.
In conclusion: I am recommended that this section be modified as follows, the discussion for this study appears to be too brief. What does the result imply? How is this consistent with other studies? The discussion should have two ways, “theoretical implications” and “practical implications.” The conclusion, limitations, and future research directions should follow the discussion section. Here, the others should highlight the study’s limitations based on the methodological design of the study. Authors should consider the generalizability of the result. Following this, future research directions can be proposed to address these limitations.
>>> We have now added new text to the Discussion and Conclusions sections to address limitations and theoretical and practical implications.
The authors need to take or cite more literature support to explain their point of view, if possible, try to add recently published work.
Finally, the language and style need to be deeply revised. I’m not a native speaker, but I found a few unclear sentences. Especially the introduction and numerical analysis (sections 3 & 4).
I wish the authors the best with this research
Round 2
Reviewer 3 Report
I thank the authors for the opportunity to review their article.
Title
>>> We have now rephrased the Title based on your suggestion OK, revised
Abstract
>>> We rewrote the abstract from scratch and integrated your suggestions. OK, now
Literature review
>>> We have now divided it according to your suggestion. OK
>>> We have now added new text to address this issue on the last paragraph. OK
>>> Supply chain definiton and explanation of topic OK
This part is partly corrected. However, Im still missing „explanation of the different factors“ (from Table 1). This should be clear, because there is no explanation why are these factors related to topic of the paper in current literature. For example Corporate Culture (how this factor influence topic from viewpoint in literature …. And other factors).
Methodology
>>> Data limitation problem OK added to conclusion part
>>> Data characteristics (small and medium sized) OK, added to text
>>> more information of the specific factors OK, very well explained
>>> focus on internal factors OK, added to text
>>> How do your selected factors OK, explained in text
Discussion
>>> added three new paragraphs in discussion section The whole discussion section has 2 paragraphs (not three new and more). I recommend enhance discussion part as authors mentioned again.
Conclusion
>>> "As stated in the Introduction" - this is an unnecessary introductory sentence. OK, revised
>>> The article lacks practical and theoretical contributions. OK, added
>>> The article does not state the shortcomings of the research. OK, added limitations
Overall this article needs still minor revision. In particular the addition of a literature review, and discussion part.
Comments for author File: Comments.pdf
Author Response
please see the attachment.
Author Response File: Author Response.docx
Reviewer 4 Report
Thank you for answering my comments and suggestions (In 1st round review report), but we can not find any improvements in the original manuscript. Please highlight your modifications with some color and provide the page numbers.
In addition, I have understood that this paper's main objective is to propose or develop a new methodology to determine the risk factors to reduce auditor subjectivity. Please highlight the limitations of present methods for this particular issue in the Introduction.
Moreover, I do not agree with your reply to Comment 4 (in the round 1 review ) that "In the literature, there is no specific method that represents a selection of risk factors. There are a few methods to finalize the factors or criteria, for example, the Delphi method.
I wish the authors the best with this research
Comments for author File: Comments.docx
Author Response
Point 1: Please highlight your modifications with some color and provide the page numbers
Response 1: We now resubmitted the paper with the main changes highlighted with a yellow background color.
Point 2: In addition, I have understood that this paper's main objective is to propose or develop a new methodology to determine the risk factors to reduce auditor subjectivity. Please highlight the limitations of present methods for this particular issue in the Introduction.
Response 2: We added a new paragraph at the end of the Introduction to refocus the reader again on the main limitation of the present method for risk assessment (see last paragraph of Introduction).
Point 3: Moreover, I do not agree with your reply to Comment 4 (in the round 1 review ) that "In the literature, there is no specific method that represents a selection of risk factors. There are a few methods to finalize the factors or criteria, for example, the Delphi method.
Response 3: We now understand the comment better and addressed it in the Introduction section (around lines 67-69) with the following text and a new reference: “While the Delphi method can be used to assess risk and reduce subjectivity, it requires averaging the assessments of a large panel of auditors (e.g., Perera et al. 2014), which obviously is not an applicable practice due to the financial costs to the firms.”
Author Response File: Author Response.docx
