Human Performance Detection Using Operator Action Log of Nuclear Power Plant

: The introduction of digital technologies into the main control room of a nuclear power plant also introduces new human errors. The operator log records the control information of operators on systems and equipment, and provides an important data source for the retrospective investigation of operating events in a nuclear power plant. A traditional operator log review is conducted manually, which has some major problems, such as being time-consuming and inefficient. This paper proposes an automatic detection method for operator logs, which models an operating procedure at three levels, including procedure, step and action. Such a model clarifies the overall logic and basic attributes of the operating procedure, and can be used as a standardized template of a control action sequence to compare with the actual operation actions in the operator log, so as to identify possible human performance deviations. This paper explains the method, and discusses the advantages and limitations of the proposed method


Introduction
Nuclear power, being a clean and safe energy source, plays a significant part in addressing the world's growing environmental concerns [1].Human error is one of the most risky factors that threatens the safe operation of nuclear power plants (NPPs) and other modern industries [2,3].Human factor issues can be found in nearly all incidents or accidents of NPPs.In recent years, more and more main control rooms (MCRs) have introduced digital technologies such as soft control, computer-based procedures (CBP) and large screen.The digital development of MCRs not only makes it convenient for operators to monitor and operate the operation information of a NPP but also introduces some new problems, such as a large number of displays, complex operations, and operators losing their position in the implementation of the procedures [4].
The nuclear industry has applied various measures to reduce and avoid human errors in MCRs.In the design phase of an NPP, the two most important measures are to develop operating procedures [5] and carry out various human factors engineering (HFE) verification and validation (V&V) activities [6].The operating procedures are a group of action sequences that are proven to be complete, mature and reliable through safety analysis and ensure that the safety state of the NPP is controllable under various specific operating conditions.In addition, it is also necessary to confirm that the operating procedures can be implemented by operators.For this purpose, some human factor reliability analysis methods, such as THERP [7], SPAR-H [8] and CREAM [9], are used to analyze and confirm that operators can complete the operation procedures within the specified time and conditions.
During the operation phase of an NPP, especially under abnormal operating conditions, operators are required to handle abnormalities in strict accordance with operating procedures.Operation experience feedback is also an important measure to continuously improve safety.Once an incident occurs, the NPP needs to submit an investigation report on the cause of the incident and the improvement or prevention measures to the governmental nuclear safety regulatory bodies.However, in recent years, it has been found that some events related to human factors have been ignored or concealed by some NPPs.One obvious reason is the concern that the incident investigation might interrupt the operation of the NPP, thus reducing the economy of the plant.Another reason is to protect operators from accountability.After all, the human error is unfavorable to the professional promotion of the operator.For this reason, in the investigation of an operation incident, human errors are usually attributed to manmachine interface (HMI) design problems or incomplete operating procedures, which is not conducive to fostering a good nuclear safety culture [10].
On the other hand, the information of most indicators (such as instruments or lights) and controls (such as buttons and knobs) in the analog MCR of NPPs is manually recorded [11].These records provide a basis for post analysis of operational events (including human error analysis).In recent years, with the digitization of the MCR, this information, together with operator control actions, has been automatically recorded by computer, which is called the operator log or operation log of the MCR.The operator log of the MCR records the operation of an NPP under normal, abnormal and accident conditions, and the amount of data is increasing with each passing day.Therefore, it is more and more difficult and time-consuming to investigate the operation events from the massive amount of operator log data.
At present, some NPPs in China have begun to consider introducing data mining technology to develop an intelligent system called an operator log audit system (OLAS) to assist in the investigation and analysis of operational events [12][13][14].The quality assurance team of an NPP uses the OLAS to regularly analyze the operating data of the NPP in a period of time (such as one week).If any operation deviation is found during the operation of the NPP (including the deviation of operator actions from the operating procedures), the quality assurance team reviews whether it is necessary to improve the plant operation.Such a positive attitude helps foster a good nuclear safety culture.
As shown in Figure 1, the operator performance can be evaluated from two dimensions: whether the operator has complied with the operating procedures and the effect of the operator's control action over the plant operation.Under these two evaluation dimensions, the operator performance level is divided into four cases.Among them, Case I is normal and in line with expectations; both Case III and Case IV are abnormal, which, respectively, reveal that the operating conditions are not covered by the operating procedures or human error has occurred.NPPs usually attach importance to the investigation and analysis of Case III and Case IV, and often ignore Case II.Case II is also normal.Although Case II does not necessarily mean that the operators have made human errors during the plant operation, the quality assurance team should pay attention to the causes of the deviations to clarify whether it is a procedure problem or a human factor problem.Although some deviations will not cause obvious operation impact, they may cause potential hazards such as increased equipment stress.In this paper, an intelligent operator performance detection method is proposed, which focuses on the automatic detection of an operator's deviation in the process of executing specific operating procedures; that is, the detection of case II and case IV.By constantly finding and solving problems through operator performance detection, the NPPs can improve operator performance and obtain a better operating experience.

Structure Analysis of Operating Procedures
The basic idea of operator performance detection is to compare a group of continuous operator actions with the operating procedures to determine whether the operator actions deviate from the operating procedures.Due to a large number of operating procedures and the large number of personnel involved in the formulation of operating procedures, the formulation of operating procedures for NPPs has not been standardized and formalized.This paper uses a unified method to describe the operating procedures, which is conducive to the modeling, verification and modification of a large number of operating procedures.
Each operation procedure is composed of initial conditions, steps and termination conditions.The initial conditions are the premise that the procedure can be executed, and the termination conditions are the criteria that the procedure has been completed.
Figure 2 presents the structure of an operating procedure "Residual Heat Removal (RHR) system startup".The procedure consists of four initial conditions: (1) The reactor coolant system is cooling through the secondary loop.
(2) The average temperature is between 160 °C and 180 °C.
(3) The pressure is between 23 bar and 27 bar.(4) At least two main pumps are running.
The termination condition of the procedure is "RHR system has been started".
Step 14 identified by the purple text box in the figure is a parallel step, which means that the parallel step shall be executed from the step "Start a RHR pump" (step 08) to the end of the last step (step 13) that is, "Adjust the manual controller to the maximum cooling rate.".Each step is also composed of three parts: (a) Conditions: conditions under which a step can be performed; (b) Action: the specific action to be performed; (c) Objective: the objective to be achieved after the execution of this step.
Unless otherwise specified, the action objective of a step is usually the action condition of the next step.In addition to the action conditions reached in the previous step, some steps also need to meet other conditions.As shown in Figure 2, the step 08 "Start a RHR pump" has a parallel step "Monitor the temperature parameters of RHR pump and motor operation" (Step 14).The parallel step is required to be performed in parallel with the step "Start a RHR pump" and its subsequent steps.Once the RHR pump temperature exceeds its specified limit, it is required to immediately terminate the execution of this procedure and start other procedures to reduce the temperature of the RHR pump.
The conditions of some special steps can also be empty, indicating that the operation step can be performed unconditionally.

Overall Structure
According to the principle of knowledge graphs and the behavior characteristics of operator monitoring and control, a Multilevel Semantic Modeling method (MSM) is proposed to model the operating procedures.As shown in Figure 3, MSM describes operating procedures at three levels: procedure, step and action.The MSM model supports top-down procedure analysis and bottom-up action evaluation.This hierarchical modeling helps to gradually clarify the semantics of the procedures described by natural language, and, on the other hand, it can also deal with the operator's action deviation layer by layer and in a standardized manner to avoid the problems of semantic ambiguity, incomplete description and logic conflict that may arise when using non-hierarchical methods such as event tree to identify the deviation.
(a) Procedure level: describes the logical structure of the operating procedure.The operating procedure is composed of steps.The logical structure of the operating procedure determines one or more procedure paths composed of steps.The operator can achieve the objective of the procedure no matter which procedure path is executed.(b) Step level: indicates the time sequence between the steps involved in completing the procedure.At the step level, the semantics of the procedure steps described in natural language are clarified, including the implementation conditions of the steps, the objectives to be achieved and the timing requirements between steps.During deviation identification, even if the operator has completed the specified steps but the conditions, objectives and timing are not met, these steps will also be identified as deviation.(c) Action level: represents the actions and objects involved in each step.Some procedures often do not clearly specify the object of an action.For example, step 01 in the procedure shown in Figure 2 "Start the 2nd pump of the two RHR pumps (001PO or 002PO)".The object of a specific action needs to be determined in combination with the actual operation status of the NPP.

Procedure Level
At the procedure level, 24 operators are used to describe the elements related to the monitoring and control behaviours of operators required by the operating procedures.As shown in Table 1, the operators are identified with different symbols and numbers.Each operator represents a human operator's behaviour.In addition to meeting the preconditions of behavior, each operator also needs to meet its own specific success criteria.Among them, operator 17 "Monitor" is a special operator, which means that monitoring is required from now on until the "End Monitor" operator is encountered.All steps (operators) during this period need to meet the success criteria specified by the monitor operator while meeting their own success criteria.
Taking the first two steps of the RHR startup process in Figure 2 as an example, the graphical model of a step at procedure level is shown in Figure 4.

Step Level
The procedure level describes the procedure through general semantics, which makes the logic of the procedure easy to understand.At the step level of an MSM model, the meaning of the procedure step represented by each operator will be clarified by using the knowledge graph (KG) method.
A knowledge graph consists of concept nodes and relation symbols [15,16].A concept node (represented by "[]") can represent an entity, attribute, state or event, corresponding to a concrete or abstract concept.For example, pumps, valves, pipelines, etc., are concrete concepts with objective entities, while temperature, pressure, flow, etc., are abstract concepts without objective entities.A concept node can represent not only the entirety, such as [pump] represents the concept of all pumps, but also actions, such as [start], [Close] and other actions.It can also represent individuals.For example, [pump: 003PO] indicates the pump numbered 003PO.A concept node can represent the quantity and constraints, for example, [pump@3] indicates that there are three pumps.Two concept nodes connected by an arrow "→" form a step model.For example, the first step of the procedure can be modeled as: [start]→[pump: active @ the 2nd pump] [stand by]→[pump: standby @ one pump] Knowledge graphs can usually be copied, restricted, connected and simplified to obtain a more simplified model.

Reasoning of a Compound Step
Most steps at the step level can be easily mapped to standard actions at the action level.However, a step at the step level may have a vague meaning or contain multiple actions (called a compound action), which are sometimes difficult for operators to accurately grasp during the plant operation, and even lead to human errors.
For example, the No. 1 step of the procedure is a compound step.In order to map a compound step into standard functions, the reasoning function of the knowledge graph can be applied.
For example, it is known that the active series include 001PO and 002PO pumps and the 001PO is in operation.The knowledge is modeled as: [active: 001PO, 002PO] [operation: 001PO] [no operation: 002PO] It is also known that the standby series include 003PO and 004PO pumps, which is modeled as: [standby: 003PO, 004PO] Then, it can be inferred that: Finally, the MSM model shown in Figure 2 is refined into that in Figure 5, which contains three standard actions, represented by operators 1-1, 1-2 and 1-3, respectively.The operators 1-4 and 1-5 describe the logic between the standard actions.The explanation of each MSM element in Table 1 is given in Table 2.

No.
Operator Explanation 0 Conditions Conditions of starting the RHR system 1-1 Step Start the 2nd RHR pump 1-2 Step Stand by the 003PO pump 1-3 Step Stand by the 004PO pump 1-4 OR Stand by either the 003PO or 004PO pump 1-5 AND Start the 002PO and stand by the 003PO or 004PO pump 2 Step Isolate the isolation valve for RHR heat exchanger cooling water

Action Level
At the action level, each operator at the step level is mapped to a standard step or combination of a group of step combinations, which are represented by action codes.Table 3 shows a group of representative action codes related to the subsequent content of this article.Using action coding, the action of an operator can be uniformly described as follows: {Action Code, Action object or objective} where "Action Code" corresponds to a detailed action of an operator, "Action object or objective" refers to the object or the desired effect of an action.For example, "stand by the 003PO pump" is coded as {0403, 003PO} where 03 indicates that the action is "Back up", and 003PO indicates that the action object is 003PO pump.
Table 4 presents the standard actions corresponding to the procedure steps in Figure 6.
Table 4. Operator explanation of the MSM model in Figure 5.

Human Performance Deviation Detection
This paper takes the operating procedure given in Figure 2 as an example to illustrate how to use the method proposed in this paper to detect human performance deviation.Figure 5 shows the MSM model at the procedure level.Except for steps 1-1 to 1-5, other steps are consistent with the procedure step number in Figure 6.

Action Sequence Identification
An MSM model at procedure level may contain one or more action sequences.In the case of multiple action sequences, the operator will select one of them to execute according to the actual plant situation.Multiple action sequences are coupled into a procedure through AND and/or OR gates.Action sequence identification is realized by decoupling the procedure.The following backtracking algorithm is used for identifying the action sequences of an operating procedure.5 presents the knowledge graph model and the action code of each step of the identified action sequences, which can be used as a standardized procedure template for human performance deviation analysis.

Human Performance Deviation Analysis
The process of human performance deviation analysis includes the following key parts: (a) The identification of executive actions: an operator log contains various I&C control signals converted from executive actions but does not contain direct evidence that the operator is required to observe, confirm and compare other behaviors in the operating procedures.Therefore, it is necessary to separate the execution operations in the operating procedures and compare them with the operator's log data.The execution actions of action sequences 1 and 2 are identified with a blue background in Table 5.As a result, a standardized procedure template is divided into an executive action template and a non-executive action template.(b) Actual action sequence detection: read the control commands from the operators in the main control room from the operator log and map them to the standard action codes.Table 6 presents an action sequence example of RHR system startup.
Selection of operating procedure: match the actual action sequence from the operator log with various standardized procedure templates to identify which operating procedure the actual action sequence corresponds to according to the degree of compliance.The degree of compliance  is defined as: where A is the total number of actions in the action sequence, and D is the number of deviation actions compared with the procedure template.
Taking the action sequence provided in Table 6 as an example, its degree of compliance with action sequence 1 and 2 in Table 5 is 90% and 85% respectively.Therefore, the action sequence 1 in Table 5 is selected as the standardized procedure template.
(d) Human performance deviation detection: the actual action sequence is compared with the standardized action sequence template.Taking the action sequence provided in Table 6 as an example, it can be seen that the action sequence from the operator log contains two redundant steps, marked with a yellow background.

Discussion
This paper presents a three-level framework for modeling and analyzing the operating procedures of NPPs.The procedure layer describes the overall logic of a procedure using unified modeling language, which not only determines the implementation conditions and purposes of the procedure but also contains the action sequences of realizing the procedure; that is, the success path.The step layer uses knowledge graphs to describe the essential attributes of the steps, and defines the operator actions, implementation conditions and objectives.The action layer uniformly encodes the operator actions to form various standardized action sequence templates for comparison and analysis with the actual operator actions.This hierarchical architecture is conducive to the analysis of operating procedures from different perspectives and removes the fuzziness caused by the non-uniform and non-standard description of the existing operating procedures.

Conclusions
The human performance deviation during the operation of an NPP can be found by mining the operator's action log.For example, a deviation in the operating procedure is identified if the operator has implemented procedures or steps that don't meeting the conditions.If the procedure termination conditions are not met, it means that the procedure goal has not been reached.Even if the initial conditions and termination conditions of the procedures are met but any intermediate step deviates, the method given in this paper will also give a prompt because any NPP requires operators to strictly follow the operating procedures.The deviation of operators from the operating procedures may reflect problems with the plant design and the formulation of operating procedures, and may also reflect problems with the nuclear safety culture of operators, which should be paid necessary attention.
One problem with operator log detection is that the scope of human performance deviation detection is limited.Because the operator log usually only contains the operator's execution actions, it can only detect the human performance deviations related to the operator's execution of a procedure or steps.Other human performance deviations involving observation, diagnosis and decision-making processes are difficult to detect directly.However, with the popularization of audio and video acquisition devices in the advanced main control room, the detection means of the human error of operators will be more abundant.

Figure 4 .
Figure 4.An example of graphical modeling operating procedures with MSM at procedure level.

Figure 5 .
Figure 5.An example of modeling operating procedures with MSM at procedure level.
(a) Select a new "End Procedure" operator as a starting point.(b) List the operators one by one forward to form a row until the "Start Procedure" operator.(c) When encountering an AND logic, list the operators on the input side horizontally.The operators with large numbers are arranged in front, and the operators with small numbers are arranged in the back.(d) When an OR gate is encountered, copy this row, paste and generate new rows downward.The number of new rows is consistent with the number of inputs of the OR gate.(e) Repeat steps (a) to (e) until all "End Procedures" have been analyzed.

Figure 6
Figure 6 contains two action sequences.Table5presents the knowledge graph model and the action code of each step of the identified action sequences, which can be used as a standardized procedure template for human performance deviation analysis.

Table 1 .
Explanation of MSM elements.

Table 2 .
Operator explanation of the MSM model in Figure2.

Table 3 .
Examples of action codes.

Table 6 .
Standard actions of the MSM model in Figure6.