Anomaly Detection for Service-Oriented Business Processes Using Conformance Analysis
Round 1
Reviewer 1 Report
Anomaly Detection for Service Oriented Business Processes Using Conformance Analysis
This paper proposed a novel technique for the identification of abnormalities in business process execution through the extension of available conformance analysis techniques. Initially, the raw event log is filtered into two variants, successful and failed, based upon the outcome of the instances. Successfully executed instances refer to the ideal conduct of the process and are utilised to discover an optimal process model. Later, the process model is used as a behavioural benchmark to classify the abnormality in the failed instances. Abnormal behaviour is compiled grounded on three dimensions of conformance, control-flow-based alignment, trace-level alignment and event-level alignment. For early predictions, we introduced the notion of conformance lifeline presenting the impact of varying fitness scores during process execution. We applied the proposed methodology to a real-world event log and presented several process-specific improvement measures in the discussion section.
1. This is an interesting piece of “Anomaly Detection for Service Oriented Business Processes” work. Please underscore the scientific value added/contributions of your paper in your abstract and introduction and address your debate shortly in the abstract.
2. Please discuss “Conformance Analysis” at the end of your introduction. Why do you proposed this method ?
3. The paper is very well structured. The material is well presented. I would suggest the author to discuss these references in your context and references. (2022) A road map for planning-deploying machine vision artifacts in the context of I ndustry 4.0, Journal of Industrial and Production Engineering, 39:3, 167-180; (2021) Sustainable industrial and operation engineering trends and challenges Toward Industry 4.0: a data driven analysis, Journal of Industrial and Production Engineering, 38:8, 581-598
4. Your conclusions' section needs to underscore the scientific value added of your paper, and/or the applicability of your findings/results, as indicated previously. Basically, you should enhance your findings, limitations, underscore the scientific value added of your paper, and/or the applicability of your contributions/shortages and future study in this session.
Author Response
Response to Reviewer-1
Thanks a lot for reviewing the paper in detail. Your comments are valuable and they really helped in improving the manuscript’s readability and neatness. All of your comments have been discussed in detail within the team and we have introduced all the required changes. Following is the comment-wise reply from our side.
Comment # 1
This is an interesting piece of “Anomaly Detection for Service Oriented Business Processes” work. Please underscore the scientific value added/contributions of your paper in your abstract and introduction and address your debate shortly in the abstract.
Highlighted lines have been added to the abstract. Line 8 & 9
Highlighted lines have been added to the Introduction section: Line 75 to 83
Comment # 2
Please discuss “Conformance Analysis” at the end of your introduction. Why do you proposed this method ?
Following lines have been added to introduction session:
Conformance analysis is the technique in process mining, during which events in the event log are replayed on to the activities in the process model. Examining the synchronisation between the recorded logs and the learned process models, conformance analysis techniques provide an in-depth analysis of real-world, complex business processes. The conventional approach to conformance analysis is the identification of deviations from routine execution; however, routine execution of a business process is not necessarily referred to as "optimal execution". Using historical logs, we determined the ideal execution of a business process so that conformance analysis techniques can accurately investigate deviations in faulty instances. This technique led to the identification of the root cause analysis for predicting business process execution abnormalities.
Comment # 3
The paper is very well structured. The material is well presented. I would suggest the author to discuss these references in your context and references. Ricardo Luhm Silva, Osiris Canciglieri Junior & Marcelo Rudek (2022) A road map for planning-deploying machine vision artifacts in the context of I ndustry 4.0, Journal of Industrial and Production Engineering, 39:3, 167-180; Ming-Lang Tseng, Thi Phuong Thuy Tran, Hien Minh Ha, Tat-Dat Bui & Ming K. Lim (2021) Sustainable industrial and operation engineering trends and challenges Toward Industry 4.0: a data driven analysis, Journal of Industrial and Production Engineering, 38:8, 581-598
Thanks for the suggestion, paper has been added. Lines 155-158
Comment # 4
Your conclusions' section needs to underscore the scientific value added of your paper, and/or the applicability of your findings/results, as indicated previously. Basically, you should enhance your findings, limitations, underscore the scientific value added of your paper, and/or the applicability of your contributions/shortages and future study in this session.
Line-661-664 have been added to the manuscript.
Individual fitness trends for process instances in the event log are examined for early detection of abnormalities in real-world business processes. The fitness trend enables failure forecasting as well as root cause investigation of issues that cause failure in a business process. Finally, root cause analysis is presented to business users in order for them to identify and correct the causes of process deviations.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
Dear Author/s, I find your study very interesting and it has been my pleasure to read it. I would suggest your paper to be published after a revision as I do have some recommendations from my end.
I would strongly recommend you to provide a discussion of your findings with practical implications and contributions.
Also, please provide limitations of your study as well as the directions for further research in the Conclusion.
Author Response
Response to Reviewer-2
Thanks a lot for reviewing the paper in detail. Your comments are valuable and they really helped in improving the manuscript’s readability and neatness. All of your comments have been discussed in detail within the team and we have introduced all the required changes. Following is the comment-wise reply from our side.
Comment # 1
Dear Author/s, I find your study very interesting and it has been my pleasure to read it. I would suggest your paper to be published after a revision as I do have some recommendations from my end.
I would strongly recommend you to provide a discussion of your findings with practical implications and contributions.
Outcomes of our proposed tehcniques were implemented at the real environment and details of the outcomes are mentioned in the Technical report. As this study is under NDA and data is protected by GDPR, the pratical implications and contributions can not be directly mentioned in this paper. Yet below mentioned text has been added at the end of the results section. Added lines:
‘As businesses strive to reduce process execution irregularities, the deviation from the ideal business process execution is not acceptable. Additionally, abnormalities in processes may use more resources than anticipated during process planning. Loss of service quality and problems with auditing and fraud are few additional effects of non-conformance in process execution. The techniques used for early prediction of business processes in this research extend the outcomes of process mining by identifying the ’outcome’ of the process at any time or phase of the process during its live execution. As a result, organisational business users can estimate whether their business processes are executing according to plan or, at the very least, according to the specifications specified in a documented standard process model.
Lines 654-662’
Comment # 2
Also, please provide limitations of your study as well as the directions for further research in the Conclusion.
Thanks for this valuable comment. Lines 684-686 have been added to the manuscript.
‘As the proposed techniques is based on historic event logs with labelled outcome, yet in future, we will explore the opportunities to include autonomous techniques for identification of success or failure of an execution business process.’
Author Response File: Author Response.pdf
Reviewer 3 Report
The paper is good and contributes to the literature. It should be publication-ready after a suitable revision.
The flow of the paper needs some work so that it's more streamlined.
The three dimensions of conformance need to be substantiated, and their selection explained.
Another case study is recommended, to demonstrate the applicability of the methodology.
The research question seems absent, and the contribution of the paper needs to be analyzed in depth.
The methodology needs to be substantiated further.
The conclusions section needs to be supplemented.
More papers can be added to the literature review.
Sources for the tables and figures need to be included.
Author Response
Response to Reviewer-3
Thanks a lot for reviewing the paper in detail. Your comments are valuable and they really helped in improving the manuscript’s readability and neatness. All of your comments have been discussed in detail within the team and we have introduced all the required changes. Following is the comment-wise reply from our side.
Comment # 1
The flow of the paper needs some work so that it's more streamlined.
The structure of the paper had been revised a few times before initial submission. But, thanks for picking up the flow-related issues. Upon your recommendation, the following changes have been produced in the manuscript.
- Sec 3.1 has been changed to Sec 3.
- ‘Checkpoints and time-based division of process’ has been moved to section ‘Data Preparation’
- ‘Experimental Setup’ has been changed to ‘Setup for Conformance Analysis’ and converved to new section.
Comment # 2
The three dimensions of conformance need to be substantiated, and their selection explained.
We have introduced several pointers within the manuscrip, along with a aparagraph at line 200-216 to explain the need of discussed techniques. Selection of the 1st Technique ‘Causality Analysis for Abnormality Detection’ is added at lines 453 – 463 .
Lines 200-216 have been added to the manuscrip which builds up the case of using the discussed three conformance analysis techniques.
Comment # 3
Another case study is recommended, to demonstrate the applicability of the methodology.
The discussion about the usability of the approach is provided in the Introduction section, lines 64-74. To detail every step for the implementation, we have covered all the steps in this research, Started from raw data collection to the final step where feedback is shared with business users. All of experiments have been conducted on real world process and results contain actual outcome which benifited the considered organization.
Comment # 4
The research question seems absent, and the contribution of the paper needs to be analyzed in depth.
Thanks for this very valuable comment. Highlighted text in Abstract , Introduction and discussion at the end has been updated to make clear the research question and solution proposed in the research.
Comment # 5
The methodology needs to be substantiated further.
Highligted Lines 183-199 have been added to the text which details about the methodology. This includes the discussion about impact of Fitness score for the abnormality deteciton.
Comment # 6
The conclusions section needs to be supplemented.
Highligted text has been added to the conclusion section.
‘Individual fitness trends for process instances in the event log are examined for early detection of abnormalities in real-world business processes. The fitness trend enables failure forecasting as well as root cause investigation of issues that cause failure in a business process. Finally, root cause analysis is presented to business users in order for them to identify and correct the causes of process deviations.’
Comment # 7
More papers can be added to the literature review.
Thanks for the suggestions, papers have been added. Lines 155-158
Comment # 8
Sources for the tables and figures need to be included.
Source of Figures 16 – 18 was missing in manuscrip and now presented at line 542.
Source of Figures 11 – 13 was missing in manuscrip and now presented at line 484.
Author Response File: Author Response.pdf
