Multi-Abnormality Attention Diagnosis Model Using One-vs-Rest Classifier in a Nuclear Power Plant
Round 1
Reviewer 1 Report
Comment 1: The motivation and the novelties of the paper are not enough to reach the quality required in this journal. Please clarify these more.
Comment 2: What is the limitation of the proposed approach in real applications?
Comment 3: The authors should consider more recent research done in the field of their study.
Comment 4: The conclusions should be extended with more discussion of future works.
Author Response
Comment 1) The motivation and the novelties of the paper are not enough to reach the quality required in this journal. Please clarify these more.
Response 1) We have followed this suggestion by adding supplementary motivation in the introduction of section 1, a new section called Concurrent occurrence of abnormalities in NPPs (section 2.1), and the conclusion.
In the introduction, we now clarify the goal of this study. This work is a prior study on an operator support system to help operators’ decision-making in multi-abnormal events.
“Research is being actively conducted on operator support systems that aid decision-making on operating procedures in NPPs [7,8]. The proposed multi-abnormality attention diagnosis model was developed as a prior study of such an operator support system. This study seeks a reduction in the operators’ workload and a reduction in human error for the accurate diagnosis of multi-abnormal transients.”
- Hsieh, M. H.; Hwang, S. L.; Liu, K. H.; Liang, S. F. M.; Chuang, C. F. A decision support system for identifying abnormal operating procedures in a nuclear power plant. Nuclear engineering and design 2012, 249, 413-418.
- Kang, J. S.; Lee, S. J. Concept of an intelligent operator support system for initial emergency responses in nuclear power plants. Nuclear Engineering and Technology 2022, 54(7), 2453-2466.
In the concurrent occurrence of abnormalities in NPPs section 2.1, we explain what abnormal operating procedures are, how to deal with them, and why multi-abnormal events are considered.
“Like other industrial plants, several types of operating procedures exist to safely operate NPPs. Nuclear operators train operating procedures to respond immediately to abnormal situations, where the operator’s action process is as follows: (1) recognizing abnormal events by alarms, (2) decision-making on the appropriate operating procedure, and (3) performing the sequences of the operating procedure. Operators recognize that the NPP has entered an abnormal state when they detect abnormal trends of the parameters through the large display panel of the MCR or when alarms go off. Following this, operators select the appropriate sub-procedure of the AOP through alarms or parameter trends numbering at least 1 to more than 20. Each sub-procedure consists of alerts and symptoms, automatic actions, countermeasures, and follow-up measures.
The aforementioned APR-1400 has about 200 sub-procedures of the AOPs, and there are thousands of indicators and many alarms. In addition, some abnormal transients require appropriate actions to be taken within a few minutes. Because of the complicated environment of abnormal transients, even well-trained operators can feel burdened and make a misdiagnosis or human error. Alarms and symptoms correspond to entry conditions, the automatic operation to check the status of the current plant variables, and depending on the conditions of the countermeasures, it may become an emergency. In the case of multi-abnormalities, the entry conditions and required tasks in the abnormal procedures are doubled within a limited time, and different entry conditions make diagnosis more difficult. In addition, there is no abnormal procedure that assumes multi-abnormalities. For these reasons, a multi-abnormality diagnosis model was developed in this study.
For instance, assume a multi-abnormality of a stuck-open pressurizer (PZR) spray valve and pilot-operated safety relief valve (POSRV). In this case, cooling water flows from the spray valve to the PZR and then from the PZR to the containment. Over time, the PZR pressure decreases and the PZR low-pressure alarm goes off. At this time, the operator recognizes the abnormal situation through the alarm and analyzes the alerts and symptoms for diagnosis. In the case of a typical abnormality of the PZR, the pressure decreases and the water level rises. But in the case of a general abnormality of the POSRV, the pressure decreases and the water level decreases. If the POSRV abnormality is more intense than that of the PZR, the water level decreases eventually. In this case the operators would check the POSRV state according to the entry conditions (alerts and alarms) and diagnose the situation as a single-abnormality of the POSRV. Operators may subsequently notice the existence of the PZR abnormality only when they have completed all countermeasures against the POSRV abnormality. As this simple example shows, overlapped abnormalities are an inherent threat to NPP operations by causing a serious failure of operator situation awareness and diagnosis. Identification of these multiple abnormalities would prevent wrong responses against the transients requiring multiplied tasks.”
In the conclusion section, we now state the novelty of the results.
“Results showed insights into which particular multi-abnormalities are hard to diagnose due to dynamic trends of the plant parameters, as found via the multi-abnormality attention function, and also demonstrated remarkable diagnostic accuracy for the target multi-abnormal events.”
We also state the clear novelty of the study in the introduction section by discussing the OVR classifier with merits in multi-label classification and the hierarchical structure of the proposed model.
In the introduction section 1,
“The developed multi-abnormality attention diagnosis model adopts a hierarchical structure using an OVR classifier since binary relevance is the most intuitive method for multi-label classification through the characteristics of independent binary learning tasks [6]. The main goals of the hierarchical structure of the proposed model are as follows: (1) providing insight into which multi-abnormal events should be trained for high accuracy in terms of data usage, (2) clustering single and multi-abnormal events, and (3) diagnosing the abnormalities. For this, methods are developed exploiting the characteristics of the difference in the predicted probability distributions between single and multi-abnormalities through a comparison of the OVR classifier with other AI models.”
- Zhang, M. L.; Li, Y. K.; Liu, X. Y.; Geng, X. Binary relevance for multi-label learning: an overview. Frontiers of Computer Science 2018, 12, 191-202.
Comment 2) What is the limitation of the proposed approach in real applications?
Response 2) From this comment, we recognized the lack of limitations of the proposed approach in real applications and have written the limitations of our work in the revised conclusion.
In the conclusion section,
“Despite the remarkable identification performance of the multi-abnormality attention diagnosis model, diagnosing multiple abnormalities in NPPs is still a challenging problem considering the highly complex and dynamic trends. To apply the proposed model to actual plants as an operator support system, the following items should be further studied. (1) In this work, we preferentially selected and tested representative system-level abnormalities. In the future, more abnormal events need to be diagnosed in consideration of high accuracy. (2) Methods to efficiently use computer resources are needed to track changing variables and diagnose various abnormalities, such as optimizing the number of hyper-parameters or separately training the multi-abnormalities with high priority. (3) Unlike real plants, simulators consist of noise-free data and can produce as much data as desired. For AI models trained with simulator data only, applying the models to real-world plants requires additional efforts to manage noise or develop robust models. If possible, it would be best to train actual plant data. (4) Following the multi-abnormality diagnosis, a systematic execution list for each abnormality is necessary to facilitate the rapid escape from the multi-abnormal events and restore safe plant operation.”
Comment 3) The authors should consider more recent research done in the field of their study.
Response 3) Thanks to this helpful comment, we considered more recent research done in the various sections.
In the introduction section,
“Research on operator support systems that support decision-making on operating procedures in NPPs is being actively conducted [7,8].
- Hsieh, M. H.; Hwang, S. L.; Liu, K. H.; Liang, S. F. M.; Chuang, C. F. A decision support system for identifying abnormal operating procedures in a nuclear power plant. Nuclear Engineering and design 2012, 249, 413-418.
- Kang, J. S.; Lee, S. J. Concept of an intelligent operator support system for initial emergency responses in nuclear power plants. Nuclear Engineering and Technology 2022, 54(7), 2453-2466.
In the related work section 2.2.,
“The FNNs have architectures of feedforward single- or multi-layer perceptrons and have been widely tested and applied [12,13].”
- El-Sefy, M.; Yosri, A.; El-Dakhakhni, W.; Nagasaki, S.; Wiebe, L. Artificial neural network for predicting nuclear power plant dynamic behaviors. Nuclear Engineering and Technology 2021, 53(10), 3275-3285.
- Ho, L. V.; Nguyen, D. H.; Mousavi, M.; De Roeck, G.; Bui-Tien, T.; Gandomi, A. H.; Wahab, M. A. A hybrid computational intelligence approach for structural damage detection using marine predator algorithm and feedforward neural networks. Computers & Structures 2021, 252, 106568.
“SVM models can solve not only multi-class classification but also multi-label classification problems using binary classification by forming a decision boundary between labels [20-22].”
- Ameer, I.; Sidorov, G.; Gomez-Adorno, H.; Nawab, R. M. A. Multi-label emotion classification on code-mixed text: Data and methods. IEEE Access 2022, 10, 8779-8789.
- Hussain, S. F.; Ashraf, M. M. A novel one-vs-rest consensus learning method for crash severity prediction. Expert Systems with Applications 2023, 228, 120443.
- Wang, Y.; Sun, P. A fault diagnosis methodology for nuclear power plants based on Kernel principle component analysis and quadratic support vector machine. Annals of Nuclear Energy 2023, 181, 109560.
Comment 4) The conclusions should be extended with more discussion of future works.
Response 4) Thank you for providing this suggestion. We now discuss our future work according to the limitations for application in real plants. Our future work is explained in three ways.
In the conclusion section,
“Despite the remarkable identification performance of the multi-abnormality attention diagnosis model, diagnosing multiple abnormalities in NPPs is still a challenging problem considering the highly complex and dynamic trends. To apply the proposed model to actual plants as an operator support system, the following items should be further studied. (1) In this work, we preferentially selected and tested representative system-level abnormalities. In the future, more abnormal events need to be diagnosed in consideration of high accuracy. (2) Methods to efficiently use computer resources are needed to track changing variables and diagnose various abnormalities, such as optimizing the number of hyper-parameters or separately training the multi-abnormalities with high priority. (3) Unlike real plants, simulators consist of noise-free data and can produce as much data as desired. For AI models trained with simulator data only, applying the models to real-world plants requires additional efforts to manage noise or develop robust models. If possible, it would be best to train actual plant data. (4) Following the multi-abnormality diagnosis, a systematic execution list for each abnormality is necessary to facilitate the rapid escape from the multi-abnormal events and restore safe plant operation.”
Reviewer 2 Report
Anomaly detection and identification is a key function of operator support system. The artificial neural networks (ANN) such as the convolutional neural network (CNN) can be applied for anomaly identification. In this paper, a set of one-vs-rest classifiers is proposed for anomaly identification, where the classifiers are implemented by the use of CNN and a newly proposed data transformation. However, the algorithm should be presented more clearly, and the analysis on the influence of newly proposed data transformation algorithm should be given. The reviewer thinks that this paper might be accepted after major revision, and the detailed comments are given as follows.
1. The selection of 60s needs to be reasonably explained. The response time for parameter changes caused by different faults should be different. Normalization is required not only on parameter values as described in equation (8), but also on time scales. Whether the different abnormal states mentioned in this article can reflect the fault characteristics within 60s?
2. The classifiers are realized based on the CNN-based algorithm proposed in [12], and a data transformation algorithm is newly proposed. It is recommended to give the structure of the CNN used for solving this anomaly diagnosis problem. Is there any influence from the data transformation algorithm to the idenfication performance?
3. Figure 2 shows that there are 3 hyperplanes are used to divide 3 classes. Ideally, OVR can perform accurate classification, but if there is noise or drift in the measured parameters, for example the zone between class 1 and class 2. How to classify the cases with indistinct features?
4. The framework of the paper in Figure 1 is not clear. After preprocessing, is the process of both sides carried out at the same time, or is the method proposed in the paper on the right and the comparison on the left? Explaining words are needed above the lines in the figure.
5. Are the parameters of OVO, CNN, ANN given in Table 2, Table 3, Table 4 as comparison methods are optimal or have been published advanced algorithms? It cannot reflect the superiority of OVR method. It is meaningless to compare the optimal OVR with the non-optimal OVO, CNN, ANN. Please add references for comparison methods (Preferably, design parameters of advanced AI algorithms published in the last 3 years).
6. The design of Table 1 is not clear. The first column should not be state but location. Need to add a third column to show how the failure changed the operating parameters of the plant. The Table 1 does not qualitatively understand the results of parameter changes caused by different fault.
7. In Section 5.2, the value of k needs to be supplemented to explain the reason for the parameter selection. Why you choose k for 2.5 and 3?
8. The figures in the paper are not clear. When inserting an image in word, the image format should be a vector image in .EMF format.
9. Is 224C2 on line 385 a mistake? The “,” in “24,976” need to be deleted.
10. ANN and CNN are not on the same level. CNN is a subset of ANN. CNN is an ANN that uses convolution operations for feature extraction. It is necessary to add which ANN method is used instead of referring to it collectively as ANN.
11. For classification, CNN can also achieve high precision, and can classify various categories, and its feature extraction ability is also strong, why must OVR be used? In addition, is the parameter setting of CNN in this paper optimal? How to explain the necessity of OVR?a
The English is acceptable.
Author Response
Response to reviewers
We are pleased to resubmit our revised manuscript. The reviewers’ comments were so valuable and helpful for further revision; we believe our manuscript has been greatly strengthened. We revised following the reviewers’ comments as below.
Please find our responses for the comments in the attached file .
Author Response File: 
Author Response.pdf
Reviewer 3 Report
The authors present a study where AI is used to develop an diagnosis model for multi-abnormal events for a nuclear power plant model. From my understanding I believe the work to be sound but it is poorly explained in places which makes it difficult to assess its impact.
I have a few concerns with the paper, firstly the paper introduction presents itself as a study where the OVR classifier is used in the training of the AI. Little justification is provided as to why this is appropriate apart from a single self-citation. However the methodology and results section clearly compared this with other methods and they chose the OVR model as it is more accurate. This is completely sound approach but not the one that is stated in the introduction.
Another thing I found very confusing is with the methodology. The authors used a simulator to obtain the training data (again this is a sound approach, but the authors do not highlight any potential issues with this data). I get that the AI can be used to obtain a model that can be used to identify all possible events. The question is given this AI model, how would this actually be used? Do the authors intend that this tool be used to identify which of the 13 states in the training set are more likely to produce abnormal events therefore training can be focused on those more likely scenarios? I'm left with the thought "so what"?
The english is ok but the descriptions are lacking some details in the introduction and conclusions. There is overuse of the phrase "abnormal event" in the introduction which is difficult to read.
Author Response
Response to reviewers
We are pleased to resubmit our revised manuscript. The reviewers’ comments were so valuable and helpful for further revision; we believe our manuscript has been greatly strengthened. We revised following the reviewers’ comments as below.
Please find our responses for the comments in the attached file .
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
The revised version can be accepted for publication.
Reviewer 2 Report
All the comments given by the reviewer have been well addressed, the paper can be accepted.
Reviewer 3 Report
The changes made make this paper acceptable for publication. I thank the authors for responding to my comments.
