A Three-Stage Dynamic Assessment Framework for Industrial Control System Security Based on a Method of W-HMM

Industrial control systems (ICS) are applied in many fields. Due to the development of cloud computing, artificial intelligence, and big data analysis inducing more cyberattacks, ICS always suffers from the risks. If the risks occur during system operations, corporate capital is endangered. It is crucial to assess the security of ICS dynamically. This paper proposes a dynamic assessment framework for industrial control system security (DAF-ICSS) based on machine learning and takes an industrial robot system as an example. The framework conducts security assessment from qualitative and quantitative perspectives, combining three assessment phases: static identification, dynamic monitoring, and security assessment. During the evaluation, we propose a weighted Hidden Markov Model (W-HMM) to dynamically establish the system’s security model with the algorithm of Baum–Welch. To verify the effectiveness of DAF-ICSS, we have compared it with two assessment methods to assess industrial robot security. The comparison result shows that the proposed DAF-ICSS can provide a more accurate assessment. The assessment reflects the system’s security state in a timely and intuitive manner. In addition, it can be used to analyze the security impact caused by the unknown types of ICS attacks since it infers the security state based on the explicit state of the system.


Introduction
The industrial control system can be remotely interacted with and communicated with cloud services [1], cyber-physical systems [2], or edge devices in a highly networked environment [3]. Cyber-attacks are increasingly becoming a threat to ICS; thus, their security is critical [4,5]. Once an essential piece of equipment experiences a safety incident, it causes a shutdown of the system and even causes casualties [6].
The security assessment of industrial control systems is an integral part of security [7,8]. The system's security integrates internal attributes and external environments. The elements involved in assessment have the following characteristics: large number, strong correlation, and poor accessibility. Thus, evaluating the security of the system accurately is difficult.
In recent years, security assessment research has mainly focused on artificial intelligence [9], medical subjects [10,11], infrastructure [12], power systems [13], coal mining [14], chemicals [15], etc. The industrial field focuses on assessing the system's functional safety or the static information security assessment for the design, and its security status is easily observerd [16,17]. However, the critical equipment executing complex control tasks in ICS is challenging for capturing security statuses directly. System security information is also related to running variables, such as system operating status and environmental status. Security assessment must have the ability to obtain security information of complex systems dynamically.
New cyberattacks for which its forms and types tend to be unknown are hard to be detected [18]. Due to the limited resources of devices and networks in ICS [19,20], it is hard to obtain security information of assessments directly by detecting attacks [21]. Cyberattacks make ICS more undependable and unsafe when using the Internet [22]. Scholars generally evaluate system security through the consequences of information attacks [23]. Consequence refers to the property losses caused by an information attack. For example, Muhammad Adil et al. identified a jamming attack channel by detecting different transmission frequencies and Round Trip Time (RTT) of transmitting a signal from multi-channel in WSNs transmission media [24].
There are three methods of assessment based on consequences of attacks [20,25]: qualitative assessment, quantitative assessment [26], and the combination of them. There are many qualitative evaluation methods, such as attack trees and fuzzy calculations. These methods are coarse-grained assessments of system parameters. For example, Xu Hui et al. used attack trees to identify various attacks for security management of SDN [27]. However, they could not provide a quantitative value to evaluate the consequences of the attack. The quantitative evaluation methods can assess the impacts of information attacks. Wenli Shang et al. provided a security assessment method based on an attack tree model with fuzzy set theory and probability risk assessment technology [28]. Jingjing Hu et al. proposed a multi-dimensional network security risk assessment framework [29], including two stages: risk identification and risk calculation. They used HMM to assess the network security risk in the risk calculation stage. The HMM assessment method can effectively reflect and quantify the security risks of the physical network system. However, they did not assign the weight to the result in the risk value calculation, as the network servers and nodes have different importance. We should weigh different parts of ICS in ICS due to its heterogeneity. Nary Subramanian et al. proposed a quantitative method of NFR (non-functional requirements) safety assessment for the infrastructure system of oil pipeline systems [30]. This method can solve the integrated assessment of functional safety and security. However, it cannot calculate the assessed value of the security. Aziz A. et al. used ontology knowledge to analyze the causal relationship between events [31], established corresponding probability models, and identified the consequences of abnormalities. This method can quantitatively assess the consequences of the system's attacks. However, the probability established by this method was stationary, while the system risk is changing. It is challenging to apply dynamic evaluations.
Currently, the most commonly used security assessment method is a combination of qualitative and quantitative information [32,33]. One is the analytic hierarchy process, which is a multi-level weight decision analysis method. Jun Chen applied the analytical hierarchy process for industrial control system evaluation [34]. Moreover, it can effectively evaluate industrial control risks. However, it cannot dynamically assess the system of ICS due to unknown attacks and threats that follow ICS. Some pieces of research had brought focus onto the necessity of a framework for the evaluation of IoT device security [35,36]. The above research methods are not dynamic and cannot meet this assessment requirement.
We propose a practical security assessment framework, a three-stage dynamic assessment framework for ICS based on a method of W-HMM. The main contributions of this work are as follows: i.
The proposed method combines a qualitative and quantitative assessment of ICS security dynamically by using a W-HMM model. The method can infer the system's risk value, which can be used as a system risk reference in a timely and intuitive manner through the explicit consequences of the attack on the device. ii. The assessment of the industrial robot control system (IRCS) is used as an example to illustrate the use of the method and compared with two typical security assessment methods.
The article is structured as follows. In the next section, we introduce the static recognition of DAF-ICSS. Dynamic monitoring is described in Section 3. Section 4 shows the assessment. Section 5 explains the framework of DAF-ICSS. Section 6 uses an IRCS as an example to verify DAF-ICSS. We discuss the results in Section 7. Finally, Section 8 summarizes the work of this paper.

Basic Value
The ICS is a part of the company's fixed assets, and its security will affect the value of the company's fixed assets. We use the analytic hierarchy process to evaluate the basic value of ICS. The basic value, B v , is used to assess the economic value of ICS. The basic value is divided into three layers: target layer, factor layer, and index layer, shown in Figure 1.
The target layer obtains B v . The factor layer decomposes the basic value of the ICS into two critical factors: asset value and asset status. Asset value represents the economic value of the ICS, and asset status reflects the state and the environment. The index layer decomposes the factors into the fine-grained index. The asset value includes three values: self-value S v , indirect value A v , and accident value A c . Self-value refers to the asset value of the ICS. Indirect value is the indirect economic loss of the enterprise caused by ICS failure without injury. accident value represents the estimated financial loss of an injury accident caused by ICS attacks. It is obtained from accident probability f 1 and accident loss g. A Asset status is divided into three states: self-state h, network state N s , and work environment E s , as shown in Table 1. The self-state reflects the performance and stability of the system, thereby affecting asset value. The latter two reflect the harshness of the system's external environment and affect the system's vulnerability value. Network state refers to the value determined by the network bandwidth, traffic, and peak value. The working environment is the value determined by temperature, humidity, and electromagnetic interference. The smaller the valuation, the lower the risk. The basic value is calculated as follows.
B One item is out of rating 0.8 Figure 1. The basic value decomposition diagram.

Vulnerability
The vulnerability V v of ICS refers to the system's weakness that attackers can abstract [37]. When the vulnerability of an ICS is attacked, a basic attack path model should be shown in Figure 2. The attack is achieved through three steps: network path connection, data manipulation, and breaking through protection. The first step ensures that the attacker can connect to ICS through the network. The second step is that the attacker sends malicious attack instructions when the attacker could imitate external communication data of ICS. The third step hides or floods attack instructions so that the instructions can pass through the protection. The attack instructions could steal, change, or delete the system's data. Moreover, the attack may cause the system's faults. The vulnerability of ICS can be illustrated from three factors depending on the three steps of the attack path model. The three factors are availability A a , data weakness S r , and safety protection S w . Availability means the degree that ICS can achieve specific operations through the network when random attacks are launched. Data weakness evaluates the possibility of communication data being attacked. Security protection is the system's ability to prevent information attacks. These three factors respectively assess the vulnerability of the three steps of the attack model. Figure 3 shows the hierarchy diagram of vulnerabilities of ICS.
The availability is divided into three indexes to measure the difficulty of attackers connecting to the system: Vector l, complexity m, and authentication r. l is used to measure the network distance between the attacker and ICS. Before an attacker can connect to the system, he must transfer instructions through network nodes such as routing equipment. The more nodes are, the more inefficiently the instructions connect. m describes the level of attack method that an attacker can achieve. When the attacker is connected to the system, r will be an index to stop the connection. The description and corresponding valuation, which are given by experts of cybersecurity, are shown in Table 2.  The data weakness is divided into three indexes: confidentiality u, integrity w, and usability v, as shown in Table 3. Safety protection S w includes three indexes: code patch C p , normal protective measure F r , and emergency protective measure S p , shown in Table 4. Code patch reflects the extent of the patches covering system vulnerabilities. Normal protective measure refers to the ability to protect the system against information attacks under normal operating conditions. The emergency protective measure is the capability to handle emergencies when in danger. The vulnerability value V v is shown as follows.

W-HMM Establishment
The HMM model can be used to build a dynamic evaluation model. Describing the stochastic process of generating explicit state sequences from hidden state sequences, HMM is a probability model related to time series. Each hidden state generates an explicit state. The security status of ICS is mostly unobservable. All the operations and the faults caused by the attack are recorded in the system's log. The security is related to the fault with a certain observation probability. The probability of mutual transition between security states is the occurrence probability. By using the occurrence probability of the security state, the current system risk probability can be calculated to monitor the system's risk dynamically.
However, HMM cannot distinguish the magnitude of the danger caused by different states. This paper proposes a W-HMM (weight HMM) method. W-HMM is the optimization method of HMM and weighs the results calculated by HMM. The value of weight is estimated based on the magnitude of the danger. The aim is to improve the accuracy of the evaluation when calculating the risk value. In W-HMM, we optimize the calculation of HMM results by weighing the security state. The W-HMM model is proposed, shown in Figure 4. We construct a mapping relation of the Markov process with parameters. The security state can be categorized into secure S 1 , monitored S 2 , attacked S 3 , and captured S 4 states. S 2 indicates that the system is scanned or spied by an attacker. In this state, the bandwidth resources are occupied, and parameters will be stolen. In the attacked state, the attacker sends malicious data, but the system has not been captured. In the captured state, the system is captured by the attacker to execute the attacker's instructions. In this state, the system may crash or perform dangerous operations. According to the severity of the fault, system faults are classified into normal O 1 , error O 2 , mild alarm O 3 , and warning O 4 ; moderate alarm O 5 ; and serious alarm O 6 , as shown in Figure 5 and Table 5.

Calculating Occurrence Probability of Security by W-HMM
The specific steps are as follows.

Constructing State and Model
The explicit state set O and the hidden state set S are, respectively, shown as follows.
The development relationships between hidden states is related by a im (1 ≤ i, m ≤ 4). a im is the probability of transition from state S i at time t to state S m at time t + 1.
The hidden state is represented by the explicit state. b ij (1 ≤ i ≤ 4, 1 ≤ j ≤ 6) is called the explicit state probability matrix, and shows the relationship between hidden and explicit state. b ij is the probability of transition from the state S i at time t to state O j at the time t + 1.
The state transition probability matrix A and the explicit state probability matrix B can be written as follows.
The W-HMM of the ICS can be described as λ, among which π represents the probability of the initial state, which is shown as follows. 2.

Algorithm of Baum-Welch
Markov model correction algorithms based on state sequences are classified into supervised and unsupervised learning algorithms [38]. A supervised learning algorithm records a large amount of state data to estimate the parameters. However, it is time consuming, costly, and causes difficulty in evaluating parameters dynamically. Unsupervised learning algorithms identify model parameters based on training samples and are suitable for calculating the parameters of W-HMM of ICS. To accurately describe the system and adapt to system changes, the W-HMM model is trained and updated iteratively with the Baum-Welch algorithm [39]. It is possible to obtain (see Appendix A) Equation (12), which represents the W-HMM model after n + 1 iterations. When adding new sample data, the current Markov parameters are taken as the initial parameters, and Baum-Welch iterative calculations are carried out to obtain the latest parameters. The occurrence probability of security risks is extracted from the state transition probability matrix of parameters.

Assessment
Field experts of ICS obtained the state weight values that affect the risk value of ICS, shown in Table 6. The risk value describes the loss caused by an attack on the company. It is equal to the product of the failure probability and consequence of the attack. The security risk value can be determined as follows.  Table 7.  Table 7. Type of parameters.

Type Identifier Description
Constant It only needs to be collected once and can be used for a long time  Figure 6, the DAF-ICSS framework is composed of static identification, dynamic monitoring, and evaluation. The framework can evaluate the security of ICS qualitatively and quantitatively. In this section, we will briefly summarize the procedure of the assessment. During data collection, there are two stages. One is static identification, which identifies the value and vulnerability of each part of the evaluation system. The analytic hierarchy process enumerates the factors that affect the value and the vulnerability of the system. The result of static identification is represented by a severity value, which is obtained by multiplying the value and vulnerability of the system.

Illustrated in
The other is dynamic monitoring, which calculates the system's security risk probability. Moreover, the calculation is based on W-HMM. Its result predicts the possibility of system security risks.
The system security state is unobservable. W-HMM is introduced for dynamic monitoring to establish the connection between the observable states and the unobservable security state. This method calculates the risk probability according to the observation state and updates the risk probability in the next new observation state. W-HMM can quantify and weigh the risk probability based on different application scenarios and models.
In the assessment stage, we obtain the system risk value by multiplying the severity value and the risk probability. We develop a risk map. The map determines the risk level by the risk value's boundary. The boundary is set by the risk tolerance of the system, shown in Figure 7. The system is safe when the risk value is in the green area. If it is in the yellow area, the system is at risk. The evaluation system will immediately issue an alarm if it reaches the red zone. Its purpose is to locate risk levels according to the calculated risk value quickly. The corresponding protection strategy can be chosen rapidly according to the level and risk value.

Experimental Setup
An attack test platform is built to verify the feasibility and effectiveness of the evaluation method proposed in this paper, shown in Figure 8. The platform comprises three computers and a robot that are connected through a gigabit router. The robot, which is ER3, is produced by Effort. The controller of the robot is Robox produced by Robox SPA.
The computers play as the remote terminal, the evaluation terminal, and the attack terminal. The remote terminal sends instructions and programs. The attack terminal attacks the control system. Meanwhile, the evaluation terminal records the robot's status data and evaluates the system.

•
Step 1: Static identification The company's asset data and data evaluated by asset management are shown in Table 8. Table 9 displays the vector and complexity in availability. When the robot connects to the PC, the robot does not perform authentications or check the content of the communication. As a result, the authentication score is 0.704. Most system security measures are warnings. The valuation of data weakness can be found in Table 9. Most system faults are alarming. When a serious failure occurs, the system will stop running. The score of the emergency protective measure is 0.5.

•
Step 2: Dynamic monitoring The initial parameters of the model are obtained by collecting and sorting empirical data. The initial parameters A 0 , B 0 , and π 0 are obtained by empirical estimation. We used two types of DOS attacks during the experiment in the two periods. In the first period, the hacker uses ping flooding random attacks. In the second period, the hacker uses UDP attacks to attack the vulnerable spots of the system. In the experiment, the attacked ports are random, and the attack frequency is once every 10 min.

•
Step 3: Assessment The failure of the industrial control system caused by the attack is probabilistic. We divide the experiment into two stages. Each attack stage lasts a week, and the robot control system status is collected once an hour during the working time every day. We record the status of the control system with three different methods: DAF-ICSS, expert [40] and HMM [41]. The data obtained by these methods are shown in Table 10. We obtain the average value of each stage evaluation from the on-site enterprise expert group. Work environment E s 0.8 Emergency protective measure S p 0.5   Figure 9 shows that the risk value evaluated by experts is close to the other algorithms at the beginning of stage 1. It means that every assessment method is accurate at the beginning. After the beginning stage, the expert's assessment remains the same at each stage and cannot dynamically evaluate the system's security. HMM and DAF-ICSS assessment methods can dynamically assess the security of the system.
At the beginning of stage 2 in the assessment, we open the UDP port of the system, which leads to a new vulnerability. We increase the weight of V v , and it results in a severe alarm state. However, the HMM assessment method draws an insensitive rise in Figure 9. Because the HMM method assigns no risk weight to a variety of system security states, DAF-ICSS can sensitively reflect the changes of system risk by weighing the evaluation elements and establishing a dynamic evaluation model. When the system is attacked by a UDP flood, the risk value assessed by DAF-ICSS exceeds CNY 40,000. The values given by other evaluation methods do not exceed CNY 40,000. DAF-ICSS assessment is more accurate than HMM. Then, the evaluation system issues an alarm and closes the UDP port to lower the risk's value. The experiment confirms the effectiveness and timeliness of DAF-ICSS.
ICS may expose different vulnerabilities during running. For example, the operator opens the remote service port by mistake, which could lead to system vulnerability. We should focus on the dynamic changes in risk value. When the risk value suddenly increases, the system is probably experiencing a security risk.
The proposed framework of DAF-ICSS is more sensitive than other methods [42,43], which could provide a dynamical risk value. Table 11 lists the comparision of some security assessment methods in performance.

Disscusion
In addition to industrial robots, the proposed method can also be used for other industrial control systems. Some parameters in the method need to be evaluated based on a specific application scenario. For example, the work environment will be changed according to different scenarios. If we obtain a more accurate risk value, the sequence and state can be increased to enhance the quality of DAF-ICSS. It will also increase the computational burden of the system.
The security assessment combines the qualitative method, which uses a risk map to determine the system's security, and the quantitative method, which uses a risk value to measure security value. Because the assessment relies on observations of ICS anomaly alarms, the risk value may be inaccurate when the alarms do not accurately match the system's state. In the future, it can be overcome to some extent.

Conclusions
Security assessment is the critical part of the system's security. We propose a security assessment method for ICS. We divide the security assessment of ICS into three steps: static recognition, dynamic monitoring, and assessment. A hierarchical system is provided for evaluating security risks. To obtain the system's risk level, the assessment method based on W-HMM calculates the industrial security risk value. It can be updated online for optimized estimation results and determine the degree of influence with different parameters in the factory. DAF-ICSS enables operators to find out a change of risk with high precision and efficiency. It also can be used to conduct cause analysis and security impact analysis.
However, this assessment method still has room for improvement, such as exploring methods for selecting more reasonable weights, etc. In addition, there is a well-known problem in industrial systems: The already designed secure architecture that does not sacrifice functionality has difficulty in providing a coordination of security and safety. In the future, we could focus on assessing multiple industrial control systems and fundamental understanding between safety and security based on dynamic security assessments so that dedicated modeling constructs and metrics can be proposed.

Conflicts of Interest:
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A
The probability of the hidden state, S i , at time, t, is expressed in Equation (A1). The probabilities of the hidden states that is S i at time t and S m at time t + 1 are defined in Equation (A2). The Baum-Welch algorithm is used to calculate the model parameters for n times, as shown in Equations (A3)-(A5). The condition for terminating iteration is that the absolute values of a n+1 ij − a n ij , b n+1 ij − b n ij or π n+1 i − π n i are within a tolerance criterion.