A Novel Network Security Risk Assessment Approach by Combining Subjective and Objective Weights under Uncertainty

Abstract

Nowadays, computer networks are playing a more and more important role in people’s daily lives. Meanwhile, the security of computer networks has also attracted widespread concern. However, up to now, there is no universal and effective assessment approach for computer network security. Therefore, a novel network security risk assessment approach by combining subjective and objective weights under uncertainty is proposed. In the proposed evaluation approach, the uncertainty of evaluation data is taken into account, which is translated into objective weights through an uncertainty measure. By combining the subjective weights of evaluation criteria and the objective weights of evaluation data, the final weights can be obtained. Then, Dempster–Shafer (D-S) evidence theory and pignistic probability transformation (PPT) are employed to derive a consensus decision for the degree of the network security risk. Two illustrative examples are given to show the efficiency of the proposed approach. This approach of risk assessment, which combines subjective and objective weights, can not only effectively evaluate computer network security, but also be widely used in decision-making.

1. Introduction

The cyber physical system is a multi-dimensional complex system that integrates computing, the network and the physical environment, and it has a wide range of applications [1,2]. When it comes to computing, many studies such as research on cloud computing have been conducted [3]. Furthermore, network security is also a rather significant component of cyber physical systems. The last few years have witnessed a burst in the exploration of network security, such as network security studies of SCADA (supervisory control and data acquisition) systems [4], the Internet of Things [5], software-defined networks [6], wireless sensor networks [7] and the smart grid [8]. Besides, many studies have researched the security of computer networks because of the emergence of a large number of cyber crimes, which are researched in many studies [9,10]. To combat cyber crimes vigorously, studies regarding computer forensics [11,12], virus prevention technologies [13], security visualization for computer network logs [14], intrusion detection [15], etc., have been performed in recent years. In addition, approaches of computer network security risk assessment are also of great significance to improve computer network security.

There is a variety of approaches to assess the security of computer networks, such as game theory [16], RBF (radial basis function) neural networks [17], attack graphs [18], vulnerability correlation graphs [19], and so on. In [20], a quantitative measure of the security risk level of networks is proposed to assess network security. Firstly, the vulnerability scanning tool is used to scan the network to determine the vulnerability of each node in the network. Then, the probability approach is employed to calculate the overall security risk level of the sub-networks and the entire network. Besides, the (fuzzy) analytic hierarchy process is also used for network security assessment [21,22,23]. In this evaluation method, the index system of network security risk assessment is first established, and then, the (fuzzy) analytic hierarchy process is applied to obtain the final evaluation results. D-S evidence theory is also an effective tool for assessing network security risk [24,25]. The index system of network security risk assessment is first needed. Then, based on the weights of indexes and the evaluation data of the bottom criteria (expressed by basic probability assignment (BPA)), D-S evidence theory is used to combine evidence from bottom to top to obtain the risk level of network security. Herein, it is worth noting that the key issue of network security risk assessment is how to deal with the uncertainty information. Many solutions such as fuzzy sets theory [26,27,28], rough sets theory [29], possibility theory [30] and D-S evidence theory [31,32,33,34] can be applied to address the problem.

However, to date, there is no universal and effective method of computer network security risk assessment. Of those studies that apply a comprehensive evaluation method to evaluate networks, only the weights of criteria are taken into account, and the weights of evaluation data are simply ignored. Therefore, a novel approach is proposed in this paper by combining subjective weights of criteria and objective weights of evaluation data under uncertainty. Based on the hierarchical structure of computer networks, subjective weights of all criteria and risk values of bottom criteria are given by experts. Then, by using an uncertainty measure $iT{U}^I$ [35], the uncertainty values of bottom evaluation data are derived. Take the reciprocal of uncertainty values, and then, normalize them to get objective weights. After that, combing the subjective and objective weights and using Dempster’s rule of combination [36], the risk values of bottom criteria are fused to be the risk values of the upper level criteria. Using the same method to combine the risk values from bottom to top and applying pignistic probability transformation (PPT) and the principle of maximum membership, the risk level of computer networks is finally derived.

The rest of the article is organized as follows. Section 2 introduces the preliminaries. Section 3 presents the network security risk assessment approach studied in [25] and the new assessment approach implemented in this paper. In Section 4, the validity and robustness of the proposed approach are examined through two numerical examples. Then, the paper is briefly concluded in Section 5.

2. Preliminaries

2.1. Dempster–Shafer Evidence Theory

D-S evidence theory [31,32,37] has many advantages in handling uncertain information and can be applied to many fields such as decision-making [38,39], risk assessment [40], reliability analysis [41], and so on. Firstly, D-S evidence theory allows probability masses to be assigned to not only singletons, but also multiple hypotheses, rather than only singleton subsets in comparison to the probability theory. Secondly, information from different sources can be combined without a prior distribution. Thirdly, instead of being forced to be assigned to some singleton subsets, a certain degree of ignorance can be allowed in some situations. A few basic concepts are introduced as follows:

Let $\mathsf{\Theta }$ be a set of mutually exclusive and collectively exhaustive events, indicated by:

$$\mathsf{\Theta }=\left\{{\theta }_1,{\theta }_2,\dots ,{\theta }_i,\dots ,{\theta }_N\right\}$$

(1)

The set $\mathsf{\Theta }$ is called a frame of discernment. The power set of $\mathsf{\Theta }$ is indicated by $2^{\mathsf{\Theta }}$, namely:

$$2^{\mathsf{\Theta }}=\left\{\mathsf{\varnothing },\left\{{\theta }_1\right\},\left\{{\theta }_2\right\},\dots ,\left\{{\theta }_N\right\},\left\{{\theta }_1,{\theta }_2\right\},\dots ,\left\{{\theta }_1,\dots ,{\theta }_i\right\},\dots ,\left\{{\theta }_1,\dots ,{\theta }_N\right\}\right\}$$

(2)

If $A\in 2^{\mathsf{\Theta }}$, A is called a proposition. In the power set $2^{\mathsf{\Theta }}$, ∅ is called the empty set, the singletons are $\left\{{\theta }_1\right\},\left\{{\theta }_2\right\},\dots ,\left\{{\theta }_N\right\}$, and the multiple hypotheses are $\left\{{\theta }_1,{\theta }_2\right\},\dots ,\left\{{\theta }_1,\dots ,{\theta }_i\right\},\dots ,\left\{{\theta }_1,\dots ,{\theta }_N\right\}$.

For a frame of discernment $\mathsf{\Theta }$, a mass function is a mapping m from $2^{\mathsf{\Theta }}$ to $[0,1]$, formally defined by:

$$m:2^{\mathsf{\Theta }}\to [0,1]$$

(3)

which satisfies the following condition:

$$m\left(\mathsf{\varnothing }\right)=0and{\displaystyle \sum _{A\in 2^{\mathsf{\Theta }}}}m\left(A\right)=1$$

(4)

In D-S evidence theory, a mass function is also called a basic probability assignment (BPA). BPA reflects the degree of support for the proposition A in the recognition framework. If $m\left(A\right)>0$, A is called a focal element, and the union is called the core of the mass function.

Associated with each BPA, the belief function $Bel$ and plausibility function $Pl$ are defined as:

$$Bel\left(A\right)={\displaystyle \sum _{B\subseteq A}}m\left(B\right)$$

(5)

$$Pl\left(A\right)=1-Bel\left(\overline{A}\right)={\displaystyle \sum _{B\cap A\ne \mathsf{\varnothing }}}m\left(B\right)$$

(6)

where $\overline{A}=\mathsf{\Theta }-A$. Obviously, $Pl\left(A\right)\ge Bel\left(A\right)$, for each $A\subseteq 2^{\mathsf{\Theta }}$.

Assume there are two BPAs indicated by $m_1$ and $m_2$; Dempster’s rule of combination is used to combine them as follows:

$$m\left(A\right)=\left\{\begin{array}{cc}\frac{1}{1-K}{\displaystyle \sum _{B\cap C=A}}{m}_1\left(B\right)m_2\left(C\right),\hfill & A\ne \mathsf{\varnothing }\hfill \\ 0,\hfill & A=\mathsf{\varnothing }\hfill \end{array}\right.$$

(7)

where:

$$K={\displaystyle \sum _{B\cap C=\mathsf{\varnothing }}}{m}_1\left(B\right)m_2\left(C\right)$$

(8)

In D-S evidence theory, K is a coefficient to measure the conflict between pieces of evidence. Note that Dempster’s rule of combination is only applicable to two such BPAs that satisfy the condition $K<1$, and there are many other combination rules [42,43]. It should also be noted that the conflict in D-S evidence theory is an open issue. Many methods have been proposed to address this issue [33,44].

2.2. Weighted Average Combination Method of Combining Mass Functions

Dempster’s rule of combination will yield counter-intuitive results when combining highly conflicting evidence. Of the alternative methods that address the problem, Murphy proposed an averaging combination method [45]. However, the weights of evidence are considered equal in this method, which does not fit most of the actual situations. Therefore, a weighted average combination method of combining mass functions was proposed in [36]. This method based on the weights of evidence considers the importance of different evidence and can efficiently handle conflicting evidence with better performance of convergence. The definition is as follows.

In a real system, the importance of each piece of evidence may be different. Suppose that there are n pieces of evidence, denoted as $m_i$, and the weight of each piece of evidence is $w_i$$(i=1,2,3,\dots .,n)$. The weighted average of evidence $\overline{m}$ is given as:

$$\overline{m}={\displaystyle \sum _{i=1}^n}(m_i\times w_i)$$

(9)

The final result can be obtained by using the classical Dempster’s rule of combination (Equations (7) and (8) to combine the weighted average of evidence $\overline{m}$$(n-1)$ times. As can be seen from Equation (9), if the weight coefficient of a piece of evidence is greater, this evidence will have a larger effect on the final combination result. On the contrary, if the weight coefficient of a piece of evidence is lower, this evidence should have a smaller effect on the final combination result.

2.3. Uncertainty Measure in D-S Evidence Theory

Uncertainty quantification of mass functions is also a crucial and open issue in D-S evidence theory. Many solutions are proposed to solve this problem such as Deng entropy [46], aggregated uncertainty $AU$ [47], the ambiguity measure $AM$ [48], uncertainty measures proposed in [49,50], and so on. In this paper, a distance-based uncertainty measure ${iTU}^I$ [35] is employed to quantify the uncertainty of mass functions in D-S evidence theory, which is an improvement of uncertainty measure ${TU}^I$ [51]. This uncertainty measure is defined as below.

Suppose that m is a BPA over FOD (frame of discernment) $\mathsf{\Theta }=\left\{{\theta }_1,{\theta }_2,\dots ,{\theta }_i,\dots ,{\theta }_n\right\}$; the total uncertainty measure for m is defined as:

$${iTU}^I\left(m\right)={\displaystyle \sum _{i=1}^n}\frac{d_E^I([0,0],[0,1])-d_E^I([Bel\left({\theta }_i\right),Pl\left({\theta }_i\right)],[0,1])}{d_E^I([0,0],[0,1])}$$

(10)

where $d_E^I$ is the Euclidean distance between two interval numbers:

$$d_E^I([a_1,b_1],[a_2,b_2])=\sqrt{{(a_1-a_2)}^2+{(b_1-b_2)}^2}$$

(11)

Here, since $d_E^I([0,0],[0,1])=1$, Equation (10) can also be written as:

$${iTU}^I\left(m\right)={\displaystyle \sum _{i=1}^n}[1-d_E^I([Bel\left({\theta }_i\right),Pl\left({\theta }_i\right)],[0,1])]$$

(12)

In this paper, the normalization is done. Namely, the total uncertainty measure for m is redefined as:

$${iTU}^I\left(m\right)=\frac{{\displaystyle \sum _{i=1}^n}[1-d_E^I([Bel\left({\theta }_i\right),Pl\left({\theta }_i\right)],[0,1])]}{n}$$

(13)

2.4. Pignistic Probability Transformation

In Smets’s transferable belief model (TBM) [52], the probability distribution $BetP$ after pignistic probability transformation (PPT) is as follows.

$$BetP\left({\theta }_i\right)={\displaystyle \sum _{B\subseteq \mathsf{\Theta }}}\frac{\left|{\theta }_i\cap B\right|}{\left|B\right|}\times m\left(B\right),\forall {\theta }_i\in \mathsf{\Theta }$$

(14)

The essence of PPT is to convert a mass function to a probability distribution. It can be seen from Equation (14) that beliefs of multiple-hypothesis focal elements are given to singletons according to the principle of equality.

3. Approach of Network Security Risk Assessment

3.1. The Network Security Risk Assessment Approach Proposed by Gao et al.

In [25], an approach for assessing network security was proposed. The specific assessment process can be divided into the following steps.

3.1.1. Establish the Index System of the Network Risk

The index system is a hierarchical structure model, which divides the factors related to network risk into three levels. The framework of the index system of network security risk assessment is shown in Figure 1. The first level of the index system is network security risk assessment, also called the target level. In the second level, there are three criteria, communication and operation, access control and assets, respectively, which are all divided into 2∼5 smaller criteria at the bottom level (see

Table 1. Criteria of the bottom level [25].

Criteria	Description of the Criteria
$a_{11}$	Prevention of Malice Software
$a_{12}$	Media Processing and Security
$a_{13}$	Operation Program and Duty
$a_{14}$	Network Management
$a_{15}$	Information and Software, Hardware Exchange
$a_{21}$	Management of Network Access
$a_{22}$	Management of User’s Access
$a_{23}$	Management of Application Access
$a_{24}$	System Access and Monitoring of Usage
$a_{31}$	Effect on Tangible Assets
$a_{32}$	Effect on Intangible Assets

).

3.1.2. Use D-S Evidence Theory to Fuse Mass Functions

In this approach, the weights of evidence are taken into account when using D-S evidence theory to fuse mass functions.

Let the set of evidence be $E=\left\{E_1,E_2,\cdots ,E_n\right\}$. The weight coefficient of evidence $E_i$ is ${\omega }_i$, where ${\omega }_i\in \left[0,1\right]$ and ${\displaystyle \sum _{i=1}^n}{\omega }_i=1$. Let ${\omega }_{max}=max\left\{{\omega }_1,{\omega }_2,\cdots ,{\omega }_n\right\}$ and the relative weight vector $W^{\prime }=\left({\omega }_1,{\omega }_2,\cdots ,{\omega }_n\right)/{\omega }_{max}$. Then, the “ratio” of BPA can be determined as ${\alpha }_i\left(0\le {\alpha }_i\le 1\right)$, where $\left(1-{\alpha }_i\right)={\omega }_i/{\omega }_{max}$, $i=1,2,\cdots ,n$.

Use the “ratio” to discount BPA. The BPA after adjustment is:

$$\left\{\begin{array}{c}{m_i}^{\prime }\left(A\right)=\left(1-{\alpha }_i\right)m_i\left(A\right),\forall A\subset \mathsf{\Theta }\hfill \\ {m_i}^{\prime }\left(\mathsf{\Theta }\right)=\left(1-{\alpha }_i\right)m_i\left(\mathsf{\Theta }\right)+{\alpha }_i\hfill \end{array}\right.$$

(15)

Suppose the risk rank of the network is divided as $X=\left\{x_1,x_2,\cdots ,x_k\right\}$. Ascertain all layer’s weights and BPA $m_{ij}\left(x_h\right)$ of the bottom layer with regard to $x_h(h=1,2,\cdots ,k)$, where $m_{ij}\left(X\right)$ represents the uncertainty. Use Equation (15) to adjust BPA, and then, use Equations (7) and (8) to combine evidence from bottom to top. Finally, BPA $m\left(x_h\right)$ of the network risk can be obtained ($h=1,2,\cdots ,k$). Note that BPA of middle level criteria still needs to be adjusted before being combined.

3.1.3. Obtain the Network Security Risk Value

After getting $m\left(x_h\right)$, the belief function $Bel\left(x_h\right)$ of network risk can be obtained by Equation (5). At last, the network security risk value can be obtained through the risk calculation formula:

$$R={\displaystyle \sum _{h=1}^k}P\left(x_h\right)Bel\left(x_h\right)$$

(16)

where $P\left(X\right)=\left\{p\left(x_1\right),p\left(x_2\right),\cdots ,p\left(x_k\right)\right\}$ represents the damage degree once the risk events happen. $p\left(x_h\right)$ represents the average value of damage degree corresponding to the risk rank $x_h$. Its range is $0\le p\left(x_h\right)\le 1$.

3.1.4. Discussion of the Work Done by Gao et al.

In the approach proposed by Gao et al., the uncertainty of BPA is measured by the probability mass assigned to the complete set, which is not a very effective quantification of uncertainty. Moreover, only the subjective weights of criteria have been considered, which makes the subjectivity of the assessment very large. Therefore, in the novel assessment approach proposed in this paper, we employ an uncertainty measure to more accurately quantify the uncertainty of BPA and transform the uncertainty into objective weights. In addition, the weighted average combination method of combining mass functions, which has a good performance of convergence, is applied to the risk assessment. The process of the novel network security risk assessment approach is shown as below.

3.2. The Novel Network Security Risk Assessment Approach Proposed in This Paper

The purpose of this paper is to propose a better approach of network security risk assessment. The process of the novel network security risk assessment approach can be divided into six steps, as depicted in Figure 2.

3.2.1. Establish a Hierarchical Structure Model

The establishment of hierarchical structure model is the premise of network security risk assessment. In this paper, there are three levels of the hierarchy structure model, as depicted in Figure 1 [25].

3.2.2. Make an Evaluation Expressed by BPA

According to the hierarchical structure of computer network security risk assessment, the evaluation of the network, specifically the risk values of bottom criteria, should be given by experts and be expressed by BPA.

3.2.3. Determine the Subjective and Objective Weights

Assume that the subjective weights of the criteria are given by experts, which are known in advance. The objective weights are determined by the uncertainty of the evaluation data. The calculation method is as follows.

Let $\mathsf{\Theta }$ = {very low (VL), low (L), middle low (ML), middle (M), middle high (MH), high (H), very high (VH)} represent seven risk levels of network security assessment. For the bottom criteria, suppose the subjective weight and BPA of the criteria $a_{ij}$ are $w_{ij}$ and $m_{ij}$, respectively ($i=1,2,3;$$j=1,2,\dots$). By using Equation (13), the uncertainty of each piece of evidence (evaluation data), denoted as $U_{ij}$ ($i=1,2,3;$$j=1,2,\dots$), can be calculated. In view of the larger uncertainty of evidence and the less useful information provided, the objective weight $u_{ij}$ can be obtained by:

$$u_{ij}=\frac{\frac{1}{U_{ij}}}{{\displaystyle \sum _{j=1}^n}\frac{1}{U_{ij}}}$$

(17)

3.2.4. Obtain Comprehensive Weights

In this part, subjective weights of criteria and objective weights of evaluation data are combined to obtain the comprehensive weights. That is to say, the final weights of bottom criteria consist of two parts: the subjective weights known in advance and the objective weights to consider the uncertainty of mass functions, which contributes to decreasing the negative influence of expert’s extreme subjectivity on the evaluation data. The comprehensive weights are indicated by:

$$z_{ij}=\frac{w_{ij}\times u_{ij}}{{\displaystyle \sum _{j=1}^n}{w}_{ij}\times u_{ij}}$$

(18)

3.2.5. Use Weighted Average Combination Rule to Combine Mass Functions

Based on the comprehensive weights $z_{ij}$ and BPAs ($m_{ij}$) of the bottom criteria, the weighted average combination rule is used to combine the evidence in this layer. When the combination of evidence is finished, the results of the combination will be regarded as the mass functions (BPAs) of the middle level criteria. Similarly, we can calculate the comprehensive weights of criteria in this layer and combine the evidence to obtain the evaluation result, which is expressed by BPA. Besides, the uncertainty of the evaluation result can also be quantified by Equation (13).

3.2.6. Obtain the Risk Level of Computer Networks

Through above five steps, the BPA of network security risk assessment can be obtained. In this paper, by Equation (14), PPT is employed to convert the mass function into a probability distribution. Then, according to the principle of maximum membership, the risk level of computer networks is finally derived.

4. Case Studies

4.1. An Example of Network Security Risk Assessment

In this subsection, a numerical example from [25] is presented to illustrate the procedure of the proposed approach of evaluating network security.

4.1.1. Establish the Hierarchical Structure of Computer Networks

Considering communication, access and assets, three parts that are of great significance to computer network security, the hierarchical structure of computer network security risk assessment is established, as shown in Figure 1 [25].

4.1.2. Make an Evaluation Expressed by BPA

The BPAs of bottom criteria are given by experts, detailed in

Table 2. The basic probability assignments (BPAs) of bottom criteria [25]: very low (VL), low (L), middle low (ML), middle (M), middle high (MH), high (H), very high (VH).

Bottom Criteria	BPA
	VL	L	ML	M	MH	H	VH	$\mathsf{\Theta }$
$a_{11}$	0	0.1	0.1	0.2	0.2	0.3	0.1	0
$a_{12}$	0	0.1	0.1	0.2	0.2	0.2	0.1	0.1
$a_{13}$	0	0.1	0.15	0.2	0.3	0.15	0	0.1
$a_{14}$	0	0.1	0.1	0.15	0.2	0.3	0.1	0.05
$a_{15}$	0.1	0.1	0.1	0.2	0.3	0.1	0	0.1
$a_{21}$	0	0	0.1	0.1	0.2	0.2	0.3	0.1
$a_{22}$	0.1	0.1	0.15	0.2	0.2	0.1	0.1	0.05
$a_{23}$	0.1	0.1	0.1	0.1	0.2	0.2	0.1	0.1
$a_{24}$	0	0.1	0.1	0.2	0.3	0.2	0.1	0
$a_{31}$	0	0.1	0.1	0.1	0.3	0.2	0.1	0.1
$a_{32}$	0	0	0.1	0.1	0.2	0.2	0.3	0.1

[25].

4.1.3. Determine the Subjective and Objective Weights

The subjective weights of criteria are given by experts in advance (see

Table 3. Subjective weights of criteria.

Middle Level Criteria	Subjective Weights	Bottom Criteria	Subjective Weights
$a_1$	0.310	$a_{11}$	0.157
		$a_{12}$	0.393
		$a_{13}$	0.164
		$a_{14}$	0.172
		$a_{15}$	0.114
$a_2$	0.580	$a_{21}$	0.281
		$a_{22}$	0.312
		$a_{23}$	0.280
		$a_{24}$	0.127
$a_3$	0.110	$a_{31}$	0.670
		$a_{32}$	0.330

[25]).

For the bottom criteria, the uncertainty of each piece of evidence, denoted as $U_{ij}$, is calculated by Equation (13), and the objective weights $u_{ij}$ are obtained by Equation (17) (see

Table 4. Objective weights of bottom criteria.

Bottom Criteria	Uncertainty Values	Objective Weights
$a_{11}$	0.1247	0.2847
$a_{12}$	0.2139	0.1660
$a_{13}$	0.2093	0.1697
$a_{14}$	0.1681	0.2112
$a_{15}$	0.2109	0.1684
$a_{21}$	0.2087	0.2063
$a_{22}$	0.1729	0.2491
$a_{23}$	0.2161	0.1993
$a_{24}$	0.1247	0.3453
$a_{31}$	0.2109	0.4974
$a_{32}$	0.2087	0.5026

).

4.1.4. Obtain Comprehensive Weights

Based on the subjective and objective weights of bottom criteria, the comprehensive weights are derived by Equation (18), detailed in

Table 5. Comprehensive weights of bottom criteria.

Bottom Criteria	Subjective Weights	Objective Weights	Comprehensive Weights
$a_{11}$	0.1570	0.2847	0.2312
$a_{12}$	0.3930	0.1660	0.3375
$a_{13}$	0.1640	0.1697	0.1440
$a_{14}$	0.1720	0.2112	0.1879
$a_{15}$	0.1140	0.1684	0.0993
$a_{21}$	0.2810	0.2063	0.2463
$a_{22}$	0.3120	0.2491	0.3302
$a_{23}$	0.2800	0.1993	0.2371
$a_{24}$	0.1270	0.3453	0.1863
$a_{31}$	0.6700	0.4974	0.6677
$a_{32}$	0.3300	0.5026	0.3323

.

4.1.5. Use Weighted Average Combination Rule to Combine the Mass Functions

On the basis of BPAs ($m_{ij}$) of bottom criteria, along with the comprehensive weights $z_{ij}$, the weighed average of evidence $\overline{m_i}$$(i=1,2,3)$ can be calculated by Equation (9). Then, we can use Equations (7) and (8) to combine the weighted average of evidence $\overline{m_1}$ 4 times to obtain $m_1$. Similarly, we can derive $m_2$ and $m_3$. Weighted average of evidence $\overline{m_i}$ and the BPAs of the bottom criteria after combination are detailed in

Table 6. The weighted average of evidence of bottom criteria.

	VL	L	ML	M	MH	H	VH	$\mathsf{\Theta }$
$\overline{m_1}$	0.0099	0.1000	0.1072	0.1906	0.2243	0.2248	0.0757	0.0675
$\overline{m_2}$	0.0567	0.0754	0.1165	0.1517	0.2186	0.1670	0.1493	0.0649
$\overline{m_3}$	0	0.0668	0.1000	0.1000	0.2668	0.2000	0.1665	0.1000

and

Table 7. The BPAs of bottom criteria after combination.

	VL	L	ML	M	MH	H	VH	$\mathsf{\Theta }$
$m_1$	0.0002	0.0227	0.0281	0.1991	0.3682	0.3713	0.0102	0.0002
$m_2$	0.0132	0.0242	0.0699	0.1431	0.4230	0.1885	0.1369	0.0012
$m_3$	0	0.0504	0.0849	0.0849	0.3524	0.2264	0.1727	0.0283

.

The BPAs of bottom criteria after combination can be viewed as the mass functions of middle level criteria. Similarly, the objective and comprehensive weights of these pieces of evidence can also be obtained (see

Table 8. Objective and comprehensive weights of middle level criteria.

Middle Level Criteria	Subjective Weights	Objective Weights	Comprehensive Weights
$a_1$	0.31	0.3692	0.3291
$a_2$	0.58	0.3487	0.5816
$a_3$	0.11	0.2821	0.0892

). Then, the weighted average of evidence $\overline{m}$ is derived. After using Equations (7) and (8) to combine $\overline{m}$ 2 times, the combination result, denoted as m, which is also the mass function of network security risk assessment (see

Table 9. The weighted average evidence and combination result of middle level criteria.

	VL	L	ML	M	MH	H	VH	$\mathsf{\Theta }$
$\overline{m}$	0.0077	0.0261	0.0575	0.1564	0.3986	0.2521	0.0984	0.0033
m	0	0.0003	0.0026	0.0468	0.7468	0.1915	0.0121	0

), is obtained. Meanwhile, the uncertainty of the evaluation result can be calculated by Equation (13), which is 0.0630 (quite small).

4.1.6. Obtain the Risk Level of Computer Networks

Through the last five steps, the evaluation result, which is expressed by BPA, is given. Applying PPT to the evaluation, the risk level of computer networks can be determined. The probability distribution after PPT is detailed in

Table 10. The probability distribution of the evaluation result.

	VL	L	ML	M	MH	H	VH
$BetP$	0	0.0003	0.0026	0.0468	0.7468	0.1915	0.0121

. According to the principle of maximum membership, the risk level of this computer network is middle high (MH).

Besides, the approach used in the study of [25] is also applied to compare with the approach proposed in this paper. As described in Figure 3, if the maximum membership principle is used to determine the risk level in these two approaches, they give the same assessment result, middle high. However, the approach proposed in this paper has a better performance of convergence, and the degree of evidence’s support for middle high (MH) is greater. More importantly, the uncertainty of the evaluation result in the study of [25] can be obtained by Equation (13), which is 0.1336, far greater than that of this paper.

Herein, we also compare and discuss the assessment of each middle level criterion by using these two assessment approach. The corresponding assessment results are shown in Table 7 and

Table 11. The BPAs of bottom criteria after combination in [25].

	VL	L	ML	M	MH	H	VH	$\mathsf{\Theta }$
$m_1$	0.0023	0.0886	0.0932	0.2082	0.2439	0.2321	0.0743	0.0575
$m_2$	0.0474	0.0516	0.1151	0.1577	0.3033	0.1656	0.1452	0.014
$m_3$	0	0.0825	0.0971	0.0971	0.3058	0.2088	0.1263	0.0825

. Using the assessment approach proposed in this paper, the uncertainty of the evaluation results of $a_1$, $a_2$ and $a_3$ can be obtained by using Equation (13), which is 0.1107, 0.1173 and 0.1450, respectively. In the approach proposed by Gao et al., the corresponding uncertainty is 0.1750, 0.1379 and 0.1953, respectively. Obviously, the use of our assessment approach can reduce the uncertainty of the assessment results. In addition, according to the evaluation data in Table 7, it can be seen that $a_1$ has the highest risk level, high (H). Therefore, more attention should be paid to $a_1$ to improve the overall network security.

All the above illustrates that the approach proposed in this paper can effectively assess the security of computer networks, which is the purpose of our study.

4.1.7. The Analysis of the Sensitivity of the Proposed Method

In this part, to examine the robustness of the proposed approach, the sensitivity analysis of the proposed approach is done by changing the BPAs of some criteria.

For example, the evidence of $a_{15}$ and $a_{22}$ (abbreviated as $m_{15}$ and $m_{22}$) is changed, respectively, by assigning all the probability mass to the complete set $\mathsf{\Theta }$, which means maximizing the uncertainty and minimizing the useful information content. Then the evaluation result is calculated. The corresponding results are shown in Figure 4 and Figure 5.

From Figure 4 and Figure 5, it can be seen that although the uncertainty of the evidence increases and the useful information content reduces, the approach proposed in this paper can still make the correct evaluation, which proves that the proposed assessment approach is robust. Besides, changes in the evidence of the criterion with the larger weight will have a greater influence on the assessment result, which accords with this fact.

4.2. Another Example of Network Security System Assessment

Herein, an example of assessing computer network security systems is presented. This assessment is implemented in [53] by using a model with two-tuple linguistic information. In this subsection, evaluation data expressed in linguistic information in [53] are converted into BPAs, and then, the novel assessment approach proposed in this paper is employed to assess network security systems.

4.2.1. Use the Assessment Approach Proposed in This Paper to Assess Network Security Systems

There are four alternative network security systems from different information technology companies, denoted as $A_i\left(i=1,2,3,4\right)$, for the military to select. The purpose of assessing these network security systems is to assist the decision-maker in making the best choice. The attributes used to evaluate these computer network security systems are denoted as $G_i\left(i=1,2,3,4,5\right)$. They are tactics, technology, economy, logistics and strategy, respectively, and their weight vector is $\omega =\left\{0.1,0.15,0.2,0.3,0.25\right\}$. There are three decision-makers, denoted as $R_i\left(i=1,2,3\right)$, and their weight vector is $\lambda =\left\{0.3294,0.3365,0.3341\right\}$. The linguistic term set S is defined as S$=\{$$s_0$ = $extremelypoor\left(EP\right)$, $s_1$ = $verypoor\left(VP\right)$, $s_2$ = $poor\left(P\right)$, $s_3$ = $medium\left(M\right)$, $s_4$ = $good\left(G\right)$, $s_5$ = $verygood\left(VG\right)$, $s_6$ = $extremelygood\left(EG\right)$}. The four possible alternatives $A_i\left(i=1,2,3,4\right)$ are to be evaluated using the linguistic term set S by the three decision-makers under the above five attributes, and construct the decision matrices $R_k={\left(r_{ij}^{\left(k\right)}\right)}_{4\times 5}\left(k=1,2,3\right)$ as follows:

$$R_1=\begin{array}{cc}& \begin{array}{ccccc}{G}_1& G_2& G_3& G_4& G_5\end{array}\\ \begin{array}{c}{A}_1\\ A_2\\ A_3\\ A_4\end{array}& \left[\begin{array}{ccccc}{S}_5& S_6& S_3& S_4& S_6\\ S_3& S_1& S_2& S_1& S_0\\ S_4& S_0& S_6& S_3& S_2\\ S_1& S_5& S_3& S_2& S_3\end{array}\right]\end{array}$$

$$R_2=\begin{array}{cc}& \begin{array}{ccccc}{G}_1& G_2& G_3& G_4& G_5\end{array}\\ \begin{array}{c}{A}_1\\ A_2\\ A_3\\ A_4\end{array}& \left[\begin{array}{ccccc}{S}_3& S_4& S_2& S_6& S_2\\ S_6& S_6& S_5& S_3& S_1\\ S_2& S_3& S_6& S_1& S_3\\ S_1& S_0& S_4& S_4& S_6\end{array}\right]\end{array}$$

$$R_3=\begin{array}{cc}& \begin{array}{ccccc}{G}_1& G_2& G_3& G_4& G_5\end{array}\\ \begin{array}{c}{A}_1\\ A_2\\ A_3\\ A_4\end{array}& \left[\begin{array}{ccccc}{S}_1& S_5& S_6& S_5& S_4\\ S_2& S_4& S_1& S_4& S_3\\ S_4& S_2& S_5& S_1& S_6\\ S_6& S_1& S_0& S_6& S_2\end{array}\right]\end{array}$$

The method to convert the decision matrices into BPAs is as follows.

• According to the weights of three decision-makers, the evaluation data based on linguistic information are transformed into the probability distribution of linguistic variables.
• By applying the uncertainty measure ${iTU}^I$, the uncertainty of the probability distribution obtained in the previous step can be derived. Then, the uncertainty is used to discount the probability distribution to generate BPAs for evaluation.

The following gives an example to clearly illustrate the process of generating BPA for evaluation.

According to the decision matrices, for $A_1$, the assessment of its desirability level under $G_1$ given by three decision-makers $R_i\left(i=1,2,3\right)$ is $s_5$, $s_3$ and $s_1$, respectively. Then, the probability distribution of $A_1$ under $G_1$ is defined as:

$$\begin{array}{c}m\left(s_1\right)=0.3341\\ m\left(s_3\right)=0.3365\\ m\left(s_5\right)=0.3294\end{array}$$

By using Equation (13), the uncertainty of the probability distribution can be calculated as 0.1091. Let $\alpha =1-0.1091=0.8909$. Then, the final BPA for evaluation is defined as:

$$\begin{array}{c}m\left(s_1\right)=\alpha \times m\left(s_1\right)=0.2976\hfill \\ m\left(s_3\right)=\alpha \times m\left(s_3\right)=0.2998\hfill \\ m\left(s_5\right)=\alpha \times m\left(s_5\right)=0.2935\hfill \\ m\left(\left\{s_1,s_3,s_5\right\}\right)=\alpha =0.1091\hfill \end{array}$$

Using the same method, the BPAs of $A_i\left(i=1,2,3,4\right)$ under attributes $G_i\left(i=1,2,3,4,5\right)$ are calculated, as shown in

Table 12. BPAs of the attributes of $A_1$.

Attributes	BPA
$G_1$	$m\left(s_1\right)=0.2976,m\left(s_3\right)=0.2998,m\left(s_5\right)=0.2935,m\left(\{s_1,s_3,s_5\}\right)=0.1091$
$G_2$	$m\left(s_4\right)=0.2998,m\left(s_5\right)=0.2976,m\left(s_6\right)=0.2935,m\left(\{s_4,s_5,s_6\}\right)=0.1091$
$G_3$	$m\left(s_2\right)=0.2998,m\left(s_3\right)=0.2935,m\left(s_5\right)=0.2976,m\left(\{s_2,s_3,s_6\}\right)=0.1091$
$G_4$	$m\left(s_4\right)=0.2935,m\left(s_5\right)=0.2976,m\left(s_6\right)=0.2998,m\left(\{s_4,s_5,s_6\}\right)=0.1091$
$G_5$	$m\left(s_2\right)=0.2998,m\left(s_4\right)=0.2976,m\left(s_6\right)=0.2935,m\left(\{s_2,s_4,s_6\}\right)=0.1091$

,

Table 13. BPAs of the attributes of $A_2$.

Attributes	BPA
$G_1$	$m\left(s_2\right)=0.2976,m\left(s_3\right)=0.2935,m\left(s_6\right)=0.2998,m\left(\{s_2,s_3,s_6\}\right)=0.1091$
$G_2$	$m\left(s_1\right)=0.2935,m\left(s_4\right)=0.2976,m\left(s_6\right)=0.2998,m\left(\{s_1,s_4,s_6\}\right)=0.1091$
$G_3$	$m\left(s_1\right)=0.2976,m\left(s_2\right)=0.2935,m\left(s_5\right)=0.2998,m\left(\{s_1,s_2,s_5\}\right)=0.1091$
$G_4$	$m\left(s_1\right)=0.2935,m\left(s_3\right)=0.2998,m\left(s_4\right)=0.2976,m\left(\{s_1,s_3,s_4\}\right)=0.1091$
$G_5$	$m\left(s_0\right)=0.2935,m\left(s_1\right)=0.2998,m\left(s_3\right)=0.2976,m\left(\{s_0,s_1,s_3\}\right)=0.1091$

,

Table 14. BPAs of the attributes of $A_3$.

Attributes	BPA
$G_1$	$m\left(s_2\right)=0.3119,m\left(s_4\right)=0.6150,m\left(\{s_2,s_4\}\right)=0.0732$
$G_2$	$m\left(s_0\right)=0.2935,m\left(s_2\right)=0.2976,m\left(s_3\right)=0.2998,m\left(\{s_0,s_2,s_3\}\right)=0.1091$
$G_3$	$m\left(s_5\right)=0.3098,m\left(s_6\right)=0.6174,m\left(\{s_5,s_6\}\right)=0.0729$
$G_4$	$m\left(s_1\right)=0.6222,m\left(s_3\right)=0.3056,m\left(\{s_1,s_3\}\right)=0.0722$
$G_5$	$m\left(s_2\right)=0.2935,m\left(s_3\right)=0.2998,m\left(s_6\right)=0.2976,m\left(\{s_2,s_3,s_6\}\right)=0.1091$

and

Table 15. BPAs of the attributes of $A_4$.

Attributes	BPA
$G_1$	$m\left(s_1\right)=0.6174,m\left(s_6\right)=0.3098,m\left(\{s_1,s_6\}\right)=0.0729$
$G_2$	$m\left(s_0\right)=0.2998,m\left(s_1\right)=0.2976,m\left(s_5\right)=0.2935,m\left(\{s_0,s_1,s_5\}\right)=0.1091$
$G_3$	$m\left(s_0\right)=0.2976,m\left(s_3\right)=0.2935,m\left(s_4\right)=0.2998,m\left(\{s_0,s_3,s_4\}\right)=0.1091$
$G_4$	$m\left(s_2\right)=0.2935,m\left(s_4\right)=0.2998,m\left(s_6\right)=0.2976,m\left(\{s_2,s_4,s_6\}\right)=0.1091$
$G_5$	$m\left(s_2\right)=0.2976,m\left(s_3\right)=0.2935,m\left(s_6\right)=0.2976,m\left(\{s_2,s_3,s_6\}\right)=0.1091$

. After getting the BPAs for evaluation, the novel assessment approach proposed in this paper is applied to assess the desirability level of network security systems. For each network security system, the subjective weights of attributes $G_i\left(i=1,2,3,4,5\right)$ are known, which is $\omega =\left\{0.1,0.15,0.2,0.3,0.25\right\}$. By using Equations (13) and (17), the corresponding objective weights are derived. Then, the comprehensive weights of attributes $G_i\left(i=1,2,3,4,5\right)$ are calculated by Equation (18) (see

Table 16. The comprehensive weights of the attributes of $A_i$.

Network Security System	Comprehensive Weights of Attributes
$A_1$	${\omega }_1=\{0.1000,0.1500,0.2000,0.3000,0.2500\}$
$A_2$	${\omega }_2=\{0.1000,0.1500,0.2000,0.3000,0.2500\}$
$A_3$	${\omega }_3=\{0.1175,0.1086,0.2360,0.3568,0.1810\}$
$A_4$	${\omega }_4=\{0.1533,0.1411,0.1882,0.2822,0.2352\}$

). Using the weighted average combination rule to combine the BPAs of these five attributes, the evaluation results of network security systems are obtained (expressed by BPA), as shown in

Table 17. The evaluation results (expressed by BPA).

Network Security System	The Evaluation Results (Expressed by BPA)
$A_1$	$m\left(s_0\right)=0.0000,m\left(s_1\right)=0.0000,m\left(s_2\right)=0.0237,m\left(s_3\right)=0.0030,$
	$m\left(s_4\right)=0.2071,m\left(s_5\right)=0.0625,m\left(s_6\right)=0.7306$
$A_2$	$m\left(s_0\right)=0.0013,m\left(s_1\right)=0.8305,m\left(s_2\right)=0.0034,m\left(s_3\right)=0.1669,$
	$m\left(s_4\right)=0.0264,m\left(s_5\right)=0.0000,m\left(s_6\right)=0.0014$
$A_3$	$m\left(s_0\right)=0.0000,m\left(s_1\right)=0.3294,m\left(s_2\right)=0.0396,m\left(s_3\right)=0.3668,$
	$m\left(s_4\right)=0.0012,m\left(s_5\right)=0.0021,m\left(s_6\right)=0.2608$
$A_4$	$m\left(s_0\right)=0.0182,m\left(s_1\right)=0.0483,m\left(s_2\right)=0.1675,m\left(s_3\right)=0.0602,$
	$m\left(s_4\right)=0.1098,m\left(s_5\right)=0.0000,m\left(s_6\right)=0.5956$

.

In order to rank these four network security systems, defuzzification is performed to get the total score for each network security system in this example. Suppose in the linguistic term set $S=\left\{s_0,s_1,s_2,s_3,s_4,s_5,s_6\right\}$, every linguistic variable is represented by a trapezoidal fuzzy number given in

Table 18. Linguistic variables for the evaluation.

Linguistic Variable	Fuzzy Numbers
$s_0$(EP)	(0,0,1,2)
$s_1$(VP)	(1,2,2,3)
$s_2$(P)	(2,3,4,5)
$s_3$(M)	(4,5,5,6)
$s_4$(G)	(5,6,7,8)
$s_5$(VG)	(7,8,8,9)
$s_6$(GP)	(8,9,10,10)

and graphically presented as Figure 6. The centroid defuzzification approach is used, and the defuzzified values for each linguistic variable are $P\left(s_i\right)=\left\{P\left(s_0\right),P\left(s_1\right),\cdots ,P\left(s_6\right)\right\}=\left\{0.7778,2,3.5,5,6.5,8,9.2222\right\}$.

After obtaining the evaluation results of network security systems expressed by BPA, PPT is carried out. In this example, since the probability mass of BPA is all assigned to singleton sets, $BetP\left(s_i\right)=m\left(s_i\right),i=0,1,\cdots ,6$. Then, the total scores for these network security systems can be obtained by Equation (19), which are 8.6818, 2.6929, 5.0611, 7.2044, respectively. Therefore, the desirability level of these four network security systems is ranked as $A_1>A_4>A_3>A_2$, and the most desirable alternative is $A_1$, which is consistent with results given in [53]. That is to say, the novel assessment approach proposed in this paper is effective and can be applied to decision-making.

$$R={\displaystyle \sum _{i=0}^6}P\left(s_i\right)\times BetP\left(s_i\right)$$

(19)

4.2.2. The Assessment of Network Security Systems by Using the Approach Proposed by Gao et al.

In this part, the assessment approach proposed in [25] is also employed to assess network security systems. The evaluation results (expressed by BPA) and the total scores of network security systems are shown in

Table 19. The evaluation results by using the assessment approach in [25].

Network Security System	The Total Score	BPA
$A_1$	7.9622	$m\left(s_4\right)=0.2930,m\left(s_5\right)=0.1082,m\left(s_6\right)=0.5630,$
		$m\left(\{s_4,s_6\}\right)=0.0126,m\left(\{s_4,s_5,s_6\}\right)=0.0231$
$A_2$	3.1515	$m\left(s_1\right)=0.6006,m\left(s_3\right)=0.2702,m\left(s_4\right)=0.0922,$
		$m\left(\{s_1,s_3\}\right)=\mathrm{0.0.0122},m\left(\{s_1,s_4\}\right)=0.0024,$
		$m\left(\{s_1,s_3,s_5\}\right)=0.0223$
$A_3$	4.0068	$m\left(s_1\right)=0.2774,m\left(s_3\right)=0.6904,$
		$m\left(\{s_1,s_3\}\right)=0.0322$
$A_4$	6.2797	$m\left(s_2\right)=0.3329,m\left(s_4\right)=0.2006,m\left(s_6\right)=0.4132,$
		$m\left(\{s_2,s_6\}\right)=0.0188,m\left(\{s_2,s_4,s_6\}\right)=0.0345$

. From the total scores given by the assessment approach proposed in [25], the desirability level of these four network security systems is ranked as $A_1>A_4>A_3>A_2$, which is also consistent with the results given in [53].

However, for $A_1$, $A_2$ and $A_4$, our novel assessment approach gives less uncertainty in the assessment results (BPA) than that of the assessment approach proposed in [25]. By using our approach, the uncertainty of the evaluation results (expressed by BPA) of $A_1$, $A_2$ and $A_4$ can be obtained by Equation (13), which is 0.0698, 0.0478 and 0.0939, respectively; while the corresponding uncertainty by using the approach proposed by Gao et al. is 0.1001, 0.0949 and 0.1195, respectively.

For $A_3$, these two assessment approaches give large differences in the assessment results (see Table 17 and Table 19). In our approach, the comprehensive weights used for evaluation are the combination of subjective weights and objective weights. It can be seen from Table 16 that after considering the uncertainty of each BPA and transforming it into objective weights, the comprehensive weights of attributes $G_2$∼$G_5$ of $A_3$ have undergone significant changes. Among them, the weights of $G_3$ and $G_4$ are significantly increased, while the weights of $G_2$ and $G_5$ are significantly reduced, which leads to the larger probability mass assigned to $s_1$ and $s_6$. That is to say, in our assessment approach, the uncertainty of the evaluation data makes the evaluation results more reasonable by adjusting the comprehensive weights. Therefore, it is more reasonable to assess $A_3$ by using the approach proposed in this paper.

4.2.3. The Ranking of Network Security Systems When Weights of Attributes Change

Herein, increase the weight (subjective weights) of $G_1$, and reduce that of $G_5$, while the weights of other attributes are unchanged, to observe the changes of the evaluation results of the network security system. The corresponding evaluation results are shown in

Table 20. The ranking of the network security system.

Weight (${\mathit{G}}_1,{\mathit{G}}_5$)	Scores (${\mathit{A}}_1,{\mathit{A}}_2,{\mathit{A}}_3,{\mathit{A}}_4$)	Ranking
(0.10,0.25)	(8.6818,2.6929,5.0611,7.2044)	$A_1>A_4>A_3>A_2$
(0.15,0.20)	(8.1930,2.7884,3.8953,5.8952)	$A_1>A_4>A_3>A_2$
(0.20,0.15)	(8.4226,3.0275,3.9050,6.1114)	$A_1>A_4>A_3>A_2$
(0.25,0.10)	(7.9419,3.5380,3.0057,5.2914)	$A_1>A_4>A_2>A_3$
(0.30,0.05)	(8.4870,4.0109,4.5264,4.2198)	$A_1>A_3>A_4>A_2$

and Figure 7.

Through Table 20 and Figure 7, we can make the following analysis.

• The score of $A_1$ fluctuates at eight points and always ranks first, indicating that $A_1$ is excellent in both $G_1$ and $G_5$.
• When the weights of $G_1$ and $G_5$ are changed, the score of $A_4$ decreases obviously. When $weight(G_1,G_5)=(0.3,0.05)$, $A_4$ ranks third, with a very low score, indicating that $A_4$ is worse in $G_1$ and that more attention should be paid to $G_1$.
• Similarly, the score of $A_3$ also decreases with the change of the weights of $G_1$ and $G_5$, which indicates that there is a larger gap between $A_3$ and $A_1$ under $G_1$ than that under $G_5$.
• The score of $A_2$ becomes higher and higher, indicating that more efforts should be made in $G_5$ to improve the overall situation of the network security system.

5. Conclusions

The contribution of this paper is to propose an effective approach of network security risk assessment. One of the crucial problems in the network security risk assessment is how to deal with uncertainty. In this paper, based on the hierarchical structure of network security risk assessment, an uncertainty measure ${iTU}^I$ is applied to quantify the uncertainty of the BPAs of criteria to obtain objective weights, and then, the comprehensive weights are obtained. Besides, the weighted average combination rule is adopted to combine the evidence from bottom to top. According to the probability distribution after using PPT and the principle of maximum membership, the risk level of computer networks can be determined.

Through analyzing the uncertainty of the evaluation results in the two illustrative examples, it is easy to find that the assessment approach proposed in this paper can significantly reduce the uncertainty of the evaluation result and give a clear and correct assessment. In addition, the second example also illustrates that our risk assessment approach of combining subjective and objective weights can be used in the decision-making field. Therefore, the novel risk assessment approach proposed in this paper is a very effective approach for assessing network security and for decision-making.