Dynamic Virtual Network Reconfiguration Method for Hybrid Multiple Failures Based on Weighted Relative Entropy

Network virtualization can offer more flexibility and better manageability for next generation Internet. With the increasing deployments of virtual networks in military and commercial networks, a major challenge is to ensure virtual network survivability against hybrid multiple failures. In this paper, we study the problem of recovering virtual networks affected by hybrid multiple failures in substrate networks and provide an integer linear programming formulation to solve it. We propose a heuristic algorithm to tackle the complexity of the integer linear programming formulation, which includes a faulty virtual network reconfiguration ranking method based on weighted relative entropy, a hybrid multiple failures ranking algorithm, and a virtual node migration method based on weighted relative entropy. In the faulty virtual network reconfiguration ranking method based on weighted relative entropy and virtual node migration method based on weighted relative entropy, multiple ranking indicators are combined in a suitable way based on weighted relative entropy. In the hybrid multiple failures ranking algorithm, the virtual node and its connective virtual links are re-embedded, firstly. Evaluation results show that our heuristic method not only has the best acceptance ratio and normal operation ratio, but also achieves the highest long-term average revenue to cost ratio compared with other virtual network reconfiguration methods.


Introduction
Network virtualization (NV) allows multiple heterogeneous virtual networks (VNs) to be embedded onto the shared substrate network (SN), providing users with a variety of network services which has become one of the most promising technologies for future Internet [1][2][3]. It can decouple the network infrastructure and network services, and allow multiple heterogeneous VNs to share the SN resources through abstraction, distribution and isolation mechanism. In recent years, NV has been widely used in various fields, such as fog-supported software defined network (SDN) [4] and virtualized networked data centers [5]. One of the most challenging problems raised in NV is the virtual network embedding (VNE) problem, which has been known as the non-deterministic polynomial hard (NP-hard) problem [6].
In recent years, the SN failure events occur frequently. SN failures degrade the service performance and reliability of VNs [7]. SN failures include node failure and link failure. In network virtualization, multiple VNs are embedded onto the same SN and share the common substrate resources. Hence, substrate node failure causes virtual node and its adjacent virtual link failure. Substrate link failure causes virtual link failure. Also, there are single node/link failures and multiple node/link failures in SN which will cause complex and multiple virtual network (VN) failures [8]. Therefore, adopting suitable survivability strategies to against the increasingly complex network failures has become one of the main challenges in VN survivability.
A number of mechanisms have been proposed to improve the VN survivability [9]. These mechanisms can be broadly classified into two categories: backup mechanism [10] and reconfiguration mechanism [11]. The backup mechanism can also be classified into dedicated backup [12] and shared backup [13]. In shared backup mechanism, the backup resources can be shared with different VNs. In the dedicated case, the backup resources only can be used for dedicated VN. However, preallocating a backup resource is extremely expensive.
Instead, the reconfiguration mechanism can re-embed the faulty part of a VN without huge backup resource consumption. In reconfiguration mechanisms, backup resources are not preallocated, and the faulty parts of VNs are re-embedded after SN failures [14]. For instance, a topology awareness VN reconfiguration algorithm is proposed in [15]. The virtual node to be migrated is mapped onto the nearest mapped substrate node which is the neighbor mapping node of the migration virtual node. This algorithm takes topology indicator into consideration and improves the resource utilization. However, resource indicator is not considered in its node re-embedding strategy. To survive link failures, a novel recovery approach to restore VN without reserving backup resources prior to embedding is proposed in [16]. It not only considers the end-delay and delay variation requirements, but also takes actions to find a constrained shortest path between the nodes utilized by the failed VNs. The resource utilization is improved. However, this algorithm cannot deal with the problem of recovering VNs from node failures. As can be seen from above two algorithms, they cannot deal with the hybrid failures which include node failures and link failures.
To survive hybrid failures in SN, a heuristic survivable VNE based on node migration and link remapping is proposed in [17]. In VNE, the artificial bee colony algorithm is proposed to achieve an optimal solution. If the substrate node fails, the failed node is migrated to a normal node based on the greed rules, and the affected links are remapped based on the shortest path algorithm. Then, the shortest path is used to provide the best bandwidth principle to remap the affected links that connect with the failed node. This algorithm can deal with hybrid failures. However, the greed rules only take resource indicator into consideration and their re-embedding performance is limited. Also, they do not consider the hybrid failures in multiple faulty VNs.
To solve the problem of recovering a batch of VNs affected by a substrate node failure, a recovery approach for maximizing recovery and minimizing the cost of recovery is designed in [18]. An integer linear programming (ILP) formulation of this recovery scheme is provided, and a fast and scalable heuristic algorithm is also proposed to tackle the computational complexity of the ILP solution. It can recover a batch of VNs affected by a substrate node failure efficiently. However, it sorts the faulty VNs and virtual nodes based on the adjacent bandwidth resource, and it ignores the topology indicators. A generalized recovery approach that can achieve customized objectives is designed, and the corresponding ILP formulation is provided in [19]. Then, a fast and scalable heuristic algorithm to tackle the computational complexity of the ILP solution is proposed, and it is demonstrated that this heuristic algorithm has good recovery performance. However, it focuses on the problem of recovering from a node failure in VNE and ranks faulty VNs based on service level agreements. It cannot deal with the problem of recovering a batch of VNs affected by hybrid failures which include node and link failures. Also, in different recovery objects, recovering the faulty VNs based on service level agreements is not suitable.
As can be seen from the current research, the VN reconfiguration mechanisms focus on taking effective measures for network recovery after failures without preallocating resources. Although numerous efforts have been devoted, there are four obvious defects in existing VN reconfiguration algorithms. At first, VN reconfiguration for different types of failures in one VN and VN reconfiguration for single type of failures in multiple VNs are both studied. However, the VN reconfiguration for multiple types of failures in multiple VNs is not studied. Secondly, Entropy 2018, 20, 711 3 of 21 in reconfiguration of multiple faulty VNs, a single indicator is selected in most faulty VN ranking strategies. In different recovery objects, these strategies are not always suitable. Thirdly, a single resource indicator or topology indicator is taken into consideration in VN re-embedding, and the performance is limited. At last, the failures are not classified into certain types, and the recovery order of different types of failures is not researched.
In this paper, an ILP formulation of VN reconfiguration is provided, firstly. Then, the heuristic algorithm, called dynamic virtual network reconfiguration method for hybrid multiple failures based on weighted relative entropy (DVNRM-HMF-WRE), is proposed to solve the ILP formulation. In DVNRM-HMF-WRE, the faulty VN reconfiguration ranking method based on weighted relative entropy (FVNRRM-WRE), hybrid multiple failures ranking algorithm (HMFRA), and virtual node migration method based on weighted relative entropy (VNMM-WRE), are proposed. At last, five comparative experiments are set to demonstrate the performance of DVNRM-HMF-WRE.
The main contributions of this paper can be summarized as follows: (1) We provide an ILP formulation of VN reconfiguration and propose the DVNRM-HMF-WRE to solve it. VNE is already intractable, and the combinatorial number of sequences of VNs in batch re-embedding further increases the complexity. Therefore, we only re-embed the faulty nodes and links without disrupting their unaffected parts in VN. (2) We introduce a weighted relative entropy (WRE) method into FVNRRM-WRE and VNMM-WRE.
In FVNRRM-WRE, we rank the faulty VNs with the help of multiple indicators, and combine them based on WRE. In VNMM-WRE, we use the WRE method to change the coefficients of node resource, and topology indicator to select the suitable candidate substrate node to reduce the resource consumption. (3) We propose the HMFRA to handle the hybrid multiple failures. In HMFRA, faulty virtual node and its connective virtual links are handled first, which are the hardest the recover. HMFRA can make full use of the limited resources and improve the recovery performance. The rest of this paper is organized as follows. Section 2 presents the models and evaluation indicators. In Section 3, we propose the ILP of VN reconfiguration. The DVNRM-HMF-WRE is given, and its details are shown in Section 4. In Section 5, we evaluate the proposed algorithm through extensive simulations and experiments. We conclude this paper with a summary and areas for future exploration in Section 6.

Models and Evaluation Indicators
In this section, we describe the models and evaluation indicators. Here, Table 1 summarizes the main notations used in this paper.   The SN is modeled as a weighted undirected graph G S = (N S , E S ), in which the substrate node set and substrate link set are represented by N S and E S , respectively. In substrate nodes, the available CPU resource of substrate node n s is denoted by cpu(n s ), and the location attribute of substrate node n s is denoted by loc(n s ). Similarly, bw(e s ) is taken to denote the available bandwidth resource of substrate link e s . Also, N Sf represents faulty substrate node set and E Sf represents the faulty substrate link set.

Virtual Network
Similar to SN, the VN can be modeled as a weighted undirected graph G V = (N V , E V ). N V represents virtual node set and E V represents the virtual link set. In virtual nodes, cpu(n v ) denotes the required CPU resource of virtual node n v . In virtual links, bw(e v ) is taken as the basic attribute to denote the required bandwidth resource of virtual link e v . T V is used to denote the VN lifetime. G Vf represents the faulty VN set. Also, N Vf represents faulty virtual node set and E Vf represents the faulty virtual link set. The SN failures are denoted by a series of failure events F i,j , I = 1,2; j = 1, 2,...,n. F 1,j denotes the substrate node failure and F 1,j ∈N Sf . F 2,j denotes the substrate link failure and F 2,j ∈E Sf , j is the failure number. t s (F i,j ) and t e (F i,j ) denote the starting and ending time of F i,j , and t s ( Also, virtual link failures can be divided into three categories. If virtual link takes faulty substrate node as the source or destination node, it is called VN link failure I. The virtual link which passes through the faulty substrate node but does not take it as the source or destination node, is called VN link failure II. The independent virtual link failure caused by independent substrate link failure is called VN link failure III.

Hybrid Multiple Failures
If multiple failures occur at the same time and belong to multiple types, they are called hybrid multiple failures.
The process of VNE and VN reconfiguration are shown in Figure 1. Figure 1a shows two VNs, and   Also, virtual link failures can be divided into three categories. If virtual link takes faulty substrate node as the source or destination node, it is called VN link failure I. The virtual link which passes through the faulty substrate node but does not take it as the source or destination node, is called VN link failure II. The independent virtual link failure caused by independent substrate link failure is called VN link failure III.

Hybrid Multiple Failures
If multiple failures occur at the same time and belong to multiple types, they are called hybrid multiple failures.
The process of VNE and VN reconfiguration are shown in Figure 1. Figure

Acceptance Ratio
The acceptance ratio (AR) is denoted by the number of VN requests which are embedded successfully, divided by the total number of VN requests. It is shown in Formula (1).
where δ is infinitely close to 0. |VN(t)| is the total number of VN requests at time t and V N map (t) is the number of VNs which are embedded successfully at time t.

Revenue to Cost Ratio
where α and β are weighting coefficients to balance CPU and bandwidth resources. In this paper, we set α = β = 1. Also, hops(e v ) is the hop counts in substrate link corresponding to virtual link e v . At time t i , if G V is faulty and can be re-embedded successfully, the revenue and cost will not change. Otherwise, the revenue and cost of G V are set to zero, and it also causes the penalty cost PCost which can be defined as where t j is terminal time of G V and µ is the penalty coefficient. In this paper, µ = 2. Long-term average revenue to cost ratio (LAR/CR) can be defined as where VN fr (t) is the set of faulty VNs which are re-embedded unsuccessfully. VN map (t) is the VNs which are embedded successfully at time t.

Normal Operation Ratio
At time t, |VN fr (t)| denotes the number of faulty VNs which are unsuccessfully re-embedded. |VN(t)| is the total number of embedded VN requests. The normal operation ratio (NOR) can be defined as

ILP of VN Reconfiguration
In this section, we formulate the ILP of VN reconfiguration. The objective function and constraints can be expressed as follows.

Objective Function
In this paper, our object is to get the maximum long-term average revenue to cost ratio. The corresponding variables in Formula (7) can be seen from Formulas (2)-(5) or Table 1.

Constraints
x(e up , e ij ) = 1 i f f e up is re-embedded onto e ij 0 otherwise . (9) ∀n In constraint (8), if virtual node n v is re-embedded onto substrate node n s , x(n v , n s ) = 1. Otherwise, x(n v , n s ) = 0. In constraint (9), if virtual link e up is re-embedded onto substrate link e ij , x(e up , e ij ) = 1. Otherwise, x(e up , e ij ) = 0. In constraint (10), if virtual node n v is faulty, s(n v ) = 1. Otherwise, s(n v ) = 0. Also, if substrate node n s is faulty, s(n s ) = 1. Otherwise, s(n s ) = 0. In constraint (11), if virtual link e v is faulty, s(e v ) = 1. Otherwise, s(e v ) = 0. If substrate link e s is faulty, s(e s ) = 1. Otherwise, s(e s ) = 0. If virtual network Constrains (12) and (13) are resource constraints, and they ensure that if virtual node n v and virtual link e up are selected to re-embed, the candidate substrate node n s and substrate link e ij should have more resources than the virtual node n v and virtual link e up request, respectively.
Constrain (15) is the connectivity constraint. It refers to the flow conservation constraint for routing one unit of net flow from virtual node n u to virtual node n p .
Constrains (16) and (17) are re-embedding constraints. Constrain (16) ensures that all virtual nodes in same VN are re-embedded on different substrate nodes. Constrain (17) ensures that each virtual node is re-embedded on up to one substrate node.
Constrains (18)- (22) are failure constraints. Constrains (18) and (19) ensure that if either the virtual node or virtual link fail in VN, the VN will also fail. Constrain (20) ensures that if one of the end-nodes in substrate link fails, the substrate link will fail. Constrain (21) ensures that virtual node and the corresponding substrate node re-embedded by it cannot fail at the same time. Constrain (22) ensures that virtual link and the corresponding substrate link re-embedded by it cannot fail at the same time.
This ILP is known to be NP-hard problem, and solving it is computationally intractable. Even though optimal results can be obtained by some exact algorithms or open source linear programming toolkit GLPK, they may incur exponential increasing running time. Consequently, they cannot be scaled to address large-scale VNs. Hence, most researchers solve the ILP problem of VN reconfiguration by proposing a corresponding heuristic algorithm which has short computational time and gets an approximate optimal solution. Therefore, we propose a novel heuristic algorithm called DVNRM-HMF-WRE to solve the ILP of VN reconfiguration.

Problem Statement
The VN reconfiguration is a complex and dynamic process. There are many event statements at the same time. We analyze the possible event statements and provide the dynamic process of VN reconfiguration first. Then, three algorithms in DVNRM-HMF-WRE are proposed. When the lifetime of VN ends, the corresponding SN resources are released.

2.
When the faulty component in SN is repaired, its resources also return to normal.

3.
When SN failure occurs, the resources of faulty component are set to zero and the VN embedded on it needs to be re-embedded. If multiple failures occur at the same time, the faulty VNs will be re-embedded one by one. When all faulty virtual components in one VN are re-embedded successfully, the VN reconfiguration is successful.

4.
When VN request arriving, VN begins to embed. If succeed, the corresponding resources are used.
When multiple SN failures occur, the process of DVNRM-HMF-WRE is shown in Figure 2. 1. When the lifetime of VN ends, the corresponding SN resources are released. 2. When the faulty component in SN is repaired, its resources also return to normal. 3. When SN failure occurs, the resources of faulty component are set to zero and the VN embedded on it needs to be re-embedded. If multiple failures occur at the same time, the faulty VNs will be re-embedded one by one. When all faulty virtual components in one VN are re-embedded successfully, the VN reconfiguration is successful. 4. When VN request arriving, VN begins to embed. If succeed, the corresponding resources are used.
When multiple SN failures occur, the process of DVNRM-HMF-WRE is shown in Figure 2. As can be seen that, from Figure 2, when failure occurs, the resources of faulty components are set to zero. If there are some VNs embedded onto the faulty components, the resources used by faulty components in VNs are released. Then, the DVNRM-HMF-WRE is used to re-embed the faulty VNs. In DVNRM-HMF-WRE, the FVNRRM-WRE is used to rank multiple faulty VNs to decide which faulty VN should be re-embedded first. The HMFRA and VNMM-WRE are used together to recover the hybrid multiple failures in one VN. The HMFRA is proposed to rank hybrid multiple faulty nodes/links in one VN, and the VNMM-WRE is provided to improve the efficiency of virtual node migration. If the VN reconfiguration is successful, the resource statement of SN I updated, otherwise, the resources used by normal components in faulty VN are released. If all VN reconfigurations are over, the DVNRM-HMF-WRE is stopped.

FVNRRM-WRE
When multiple failures occur at the same time, and substrate resources are limited, the reconfiguration sequence of faulty VNs is especially important. In faulty VN ranking strategies, As can be seen that, from Figure 2, when failure occurs, the resources of faulty components are set to zero. If there are some VNs embedded onto the faulty components, the resources used by faulty components in VNs are released. Then, the DVNRM-HMF-WRE is used to re-embed the faulty VNs. In DVNRM-HMF-WRE, the FVNRRM-WRE is used to rank multiple faulty VNs to decide which faulty VN should be re-embedded first. The HMFRA and VNMM-WRE are used together to recover the hybrid multiple failures in one VN. The HMFRA is proposed to rank hybrid multiple faulty nodes/links in one VN, and the VNMM-WRE is provided to improve the efficiency of virtual node migration. If the VN reconfiguration is successful, the resource statement of SN I updated, otherwise, the resources used by normal components in faulty VN are released. If all VN reconfigurations are over, the DVNRM-HMF-WRE is stopped.

FVNRRM-WRE
When multiple failures occur at the same time, and substrate resources are limited, the reconfiguration sequence of faulty VNs is especially important. In faulty VN ranking strategies, ranking indicators and ranking method are very important. Among them, faulty component number, VN revenue, and remaining lifetime of the faulty VN all have obvious impact on the reconfiguration performance. They are selected to rank the faulty VNs. When ranking faulty VNs, a fixed formula is usually used, and it cannot adapt to the changing environment. Therefore, the WRE method is introduced in this paper.
The WRE method is a classical ranking method which can adjust the indicator coefficients to adapt the changing environment. It is improved by the classical technique for order preference by similarity to an ideal solution (TOPSIS) [20]. In TOPSIS, the generalized distance is unable to distinguish the point in perpendicular bisector between positive and negative ideal solution [21]. Therefore, the relative entropy is used, instead, to the generalized distance.
For system A and B in state A i and B i (i = 1, 2, . . . ,N), their relative entropy, C, can be defined by There are N faulty VNs, and each faulty VN has M evaluation indicators. The j-th indicator coefficient of the i-th node is denoted as X ij (i = 1, 2, . . . , N; j = 1, 2, . . . , M), and all coefficients of faulty VNs constitute a decision matrix X which is denoted as Also, the coefficient is normalized to make a fair comparison between different indicators. In different environments, each indicator has a different importance. The weighting coefficient of the j-th indicator is expressed as ω j (j = 1, 2, . . . , M, ∑ ω j = 1), and the weighted normalized decision matrix is denoted by The positive and negative ideal solutions A + and A − are determined as The relative entropies of each solution to positive and negative ideal solution are calculated as Entropy 2018, 20, 711

of 21
The similarity between each solution and the ideal one is calculated as The details of FVNRRM-WRE method are described in Algorithm 1. In Algorithm 1, X is the multiple indicators decision matrix which includes three indicators, called faulty components number, VN revenue, and remaining lifetime of the faulty VN. To calculate Z i , which is the importance degree of faulty VN i , the WRE method is widely used based on Formulas (26)-(30). After getting the importance degrees of faulty VNs, rank the faulty VNs in order of importance degree, from large to small, which can re-embed the important VN first.

HMFRA
In a faulty VN, there are multiple failures which include node failures and link failures. Recovery sequence is important in VN reconfiguration, and the HMFRA is proposed to recover different types of failures in a suitable manner. The details of HMFRA are shown in Algorithm 2.
As can be seen from Algorithm 2: 1.
Faulty virtual node and its connective links (VN link failures I) are recovered first. If substrate node is faulty, virtual node and its connective virtual links both need to be re-embedded. They will consume huge substrate node and link resources. They are the hardest to re-embed. if all virtual nodes are migrated and all their adjacent links are re-embedded successfully 4.
Update G S and G V 5.
if the set of VN link failure II is not empty 6.
Re-embed all VN link failures II 7.
if all VN link failures II are re-embedded successfully 8.
Update G S and G V 9.
if the set of VN link failure III is not empty 10.
Re-embed all of them 11.
if all VN link failures III are re-embedded successfully 12.
Update G S and G V 13.
else release the resources in G S which are used by the normal parts 15.
Update G S 16. Return

VNMM-WRE
In HMFRA, the method of faulty virtual node migration is called VNMM-WRE, which is proposed in this section and the method of virtual link re-embedding is k-shortest path algorithm [22]. Similar to FVNRRM-WRE, the WRE method is also introduced into VNMM-WRE. In virtual node ranking, the importance degree of each virtual node Z(n v ) is calculated using WRE method, which selects node CPU resource and node adjacent link bandwidth resource as the indicators. In substrate node ranking, the importance degree of each substrate node Z(n s ) is calculated using WRE method which selects node CPU resource, node adjacent link bandwidth resource and the reciprocal of hop counts between candidate substrate node and substrate node which is embedded by the neighboring virtual node of the migrated virtual node as the indicators [23].
The details of VNMM-WRE are shown in Algorithm 3

Algorithm 3. The VNMM-WRE
Input: Faulty virtual nodes which need to migrate Output: NodeMigraingList 1. for each virtual node n v ∈N V 2.
Calculate the Z(n v ) 3. end for 4. Rank virtual nodes in order of Z(n v ) from large to small 5. Save virtual node ranking order into VirtualNodeList 6. for each virtual node in VirtualNodeList 7.
Select the candidate substrate node set Can(n vi ) which satisfies the resource constraints 8.
Remove the embedded substrate nodes from Can(n vi ) 9.
if Can(n vi ) is empty 10.
for each candidate node n s in Can(n vi ) 13.
Calculate the Z(n s ) of each substrate node in Can(n vi ) 14.
end for 15.
for each virtual node in Can(n vi ) 16.
Re-embed n v to n s which has the largest Z(n s ) value 17.
Re-embed the virtual links which connect the virtual node 18.
if all virtual links are re-embedded successfully 19.
Save the substrate node n s into NodeMigratingList 20. break 21. else 22.
Remove the virtual node n v from Can(n vi ) 23. end if 24. end for 25.
Update G S and G V 26.

1.
Lines 1-5: calculate the Z(n v ) of each faulty virtual node and rank them based on Z(n v ). Then save the ranking results into VirtualNodeList.

2.
Lines 6-14: select the candidate substrate nodes and calculate their Z(n s ).

3.
Lines 15-28: re-embed the virtual node and its adjacent virtual links.
In summary, with the help of FVNRRM-WRE, HMFRA, and VNMM-WRE, the faulty VNs caused by hybrid multiple failures can be recovered.

COMPLEXITY ANALYSIS
The DVNRM-HMF-WRE includes FVNRRM-WRE, HMFRA, and VNMM-WRE. In FVNRRM-WRE, all faulty VNs are ranked based on the WRE method. Its complexity is O (|N FN |), in which |N FN | is the total number of faulty VNs. The HMFRA and VNMM-WRE are used, together, to recover the hybrid multiple failures in VNs. In the recovery of faulty VNs, the complexity of virtual node re-embedding is O(|N V ||N S | 2 ). |N V | is the total number of faulty virtual nodes in all faulty VNs and |N S | is the total number of substrate nodes. The virtual link re-embedding algorithm is k-shortest path algorithm and its complexity is O(k(|N V | + |E V |)(|E S | + |N S |lg|N S |)). |E V | represents the total number of faulty virtual links in all faulty VNs and |E S | represents the total number of substrate links. Therefore, the total complexity of DVNRM-HMF-WRE is

Simulation
In this section, we present the performance evaluation of the proposed DVNRM-HMF-WRE.

Simulation Setup
We run our experiments on a workstation with Lenovo Tianyi 510Pro (Lenovo, Beijing, China) with Windows 10 system (Microsoft Corporation, Redmond, WA, USA). The hardware platform is the Inter Core i7-7700 3.6 GHz process (Intel Corporation, Santa Clara, CA, USA) with 8 GB RAM (Intel Corporation, Santa Clara, CA, USA). The software is MATLAB R2007a (MathWorks, Natick, MA, USA). We run our simulations for 3000 time units to leave the performance in a stable state. Each simulation is performed 10 times, and we take the average values as the final results. Simulation codes and results could be found in the Supplementary File.
The SN topology and VN topology used in simulation are generated by the improved Salam network topology random generation algorithm [24]. It is worth noting that, the main parameters of VN, SN, and failures are summarized in Table 2. In Table 2, [xmin, xmax] denotes a uniform distribution between xmin and xmax. Pois{p} and Expo{g} stand for the Poisson and exponential distributions, with mean p and g, respectively.

Scenarios
Five scenarios are set in our simulation to validate the performance of DVNRM-HMF-WRE.
In the first scenario, the performance of DVNRM-HMF-WRE is compared with two typical VN reconfiguration methods TA-VNR [15] and ReNoVatE [18].
In the second scenario, our FVNRRM-WRE is compared with the other three faulty VN ranking methods to evaluate the impact of the FVNRRM-WRE. The details of them are shown in Table 3. In the third scenario, our HMFRA is compared with the other two failure handling methods to evaluate the impact of the HMFRA. The details of them are shown in Table 4. In the fourth scenario, our VNMM-WRE is compared with the other two VN node migration methods which include DVNRM-HMF-HOPS and DVNRM-HMF-RES. The details of them are shown in Table 5. Table 5. Comparison of VN node migration methods.

Algorithm Description
DVNRM-HMF-WRE VN reconfiguration method proposed in this paper, in which virtual node is migrated based on VNMM-WRE.
DVNRM-HMF-RES VN reconfiguration method in which virtual node is migrated based on the total node and link resources.

DVNRM-HMF-HOPS
VN reconfiguration method in which virtual node is migrated based on the reciprocal of hop counts between candidate substrate node and substrate node which is embedded by the neighboring virtual node of the migrated virtual node.
At last, the arrival rate of failure, u, and the lifetime of failure, T, are set to evaluate their effects on DVNRM-HMF-WRE.

Comparison of Different VN Reconfiguration Methods
The results of comparison of faulty VN reconfiguration methods are shown in Figure 3.

Comparison of Different VN Reconfiguration Methods
The results of comparison of faulty VN reconfiguration methods are shown in Figure 3. As can be seen from Figure 3a-c, the DVNRM-HMF-WRE proposed in this paper has the best AR, NOR, and LAR/CR. It achieves 5.7% and 11.3% higher AR than TA-VNR and ReNoVatE, respectively. Its NOR is 6.9% and 11% higher than TA-VNR and ReNoVatE, respectively. Its LAR/CR is 25.7% and 41.9% higher than TA-VNR and ReNoVatE, respectively. In DVNRM-HMF-WRE, faulty components number, VN revenue, and remaining lifetime of faulty VN are all considered to rank the faulty VNs. The resource and topology indicators are both used based on WRE in VNMM-WRE. In addition, the faulty virtual node and VN link failures I are recovered first. Therefore, the performance of DVNRM-HMF-WRE is the best.

Comparison of Faulty VN Ranking Methods
The results of comparison of faulty VN ranking methods are shown in Figure 4. As can be seen from Figure 3a-c, the DVNRM-HMF-WRE proposed in this paper has the best AR, NOR, and LAR/CR. It achieves 5.7% and 11.3% higher AR than TA-VNR and ReNoVatE, respectively. Its NOR is 6.9% and 11% higher than TA-VNR and ReNoVatE, respectively. Its LAR/CR is 25.7% and 41.9% higher than TA-VNR and ReNoVatE, respectively. In DVNRM-HMF-WRE, faulty components number, VN revenue, and remaining lifetime of faulty VN are all considered to rank the faulty VNs. The resource and topology indicators are both used based on WRE in VNMM-WRE. In addition, the faulty virtual node and VN link failures I are recovered first. Therefore, the performance of DVNRM-HMF-WRE is the best.

Comparison of Failure Handling Methods
The results of comparison of failure handling methods are shown in Figure 5.
revenue, remaining lifetime of the faulty VN, and re-embedding the faulty VN which has higher penalty cost and revenue based on WRE, can improve the LAR/CR significantly.

Comparison of Failure Handling Methods
The results of comparison of failure handling methods are shown in Figure 5. As can be seen from Figure 5a-c, the performance of DVNRM-HMF-WRE is better than the other two failure handling methods. It achieves 5.7% and 7.3% higher AR than DVNRM-HMF-WRE II and DVNRM-HMF-WRE III, respectively. Its NOR is 8.5% and 12.8% higher than DVNRM-HMF-WRE II and DVNRM-HMF-WRE III, respectively. Its LAR/CR is 24% and 27.9% higher than DVNRM-HMF-WRE II and DVNRM-HMF-WRE III, respectively. When the substrate node is faulty, the virtual node and its connective virtual links (VN link failure I) will fail at the same time. Substrate node and link resources are both consumed to re-embed the faulty VN. Compared with the other two link failures, node failure is the hardest to recover. Therefore, re-embedding faulty virtual node and its connective virtual links, first, can improve the performance of VN recovery algorithm.

Comparison of Different VN Node Migration Methods
The results of comparison of VN node migration methods are shown in Figure 6. As can be seen from Figure 5a-c, the performance of DVNRM-HMF-WRE is better than the other two failure handling methods. It achieves 5.7% and 7.3% higher AR than DVNRM-HMF-WRE II and DVNRM-HMF-WRE III, respectively. Its NOR is 8.5% and 12.8% higher than DVNRM-HMF-WRE II and DVNRM-HMF-WRE III, respectively. Its LAR/CR is 24% and 27.9% higher than DVNRM-HMF-WRE II and DVNRM-HMF-WRE III, respectively. When the substrate node is faulty, the virtual node and its connective virtual links (VN link failure I) will fail at the same time. Substrate node and link resources are both consumed to re-embed the faulty VN. Compared with the other two link failures, node failure is the hardest to recover. Therefore, re-embedding faulty virtual node and its connective virtual links, first, can improve the performance of VN recovery algorithm.

Comparison of Different VN Node Migration Methods
The results of comparison of VN node migration methods are shown in Figure 6. As can be seen from Figure 6a,b, DVNRM-HMF-WRE has the best AR and NOR. It achieves approximately 8.3% and 16.3% higher AR than VNMM-HMF-HOPS and VNMM-HMF-RES, respectively. The NOR of VNMM-HMF-WRE is approximately 7.4% and 17.4% higher than VNMM-HMF-HOPS and VNMM-HMF-RES, respectively. In Figure 6c, the advantage of VNMM-WRE is more obvious in LAR/CR. It is approximately 31.8% and 68.3% higher than VNMM-HMF-HOPS and VNMM-HMF-RES, respectively. In VNMM-HMF-WRE, the resource and topology indicators are both used to reduce substrate resource consumption in VN reconfiguration. Also, WRE is introduced to calculate the candidate substrate node importance.  As can be seen from Figure 6a,b, DVNRM-HMF-WRE has the best AR and NOR. It achieves approximately 8.3% and 16.3% higher AR than VNMM-HMF-HOPS and VNMM-HMF-RES, respectively. The NOR of VNMM-HMF-WRE is approximately 7.4% and 17.4% higher than VNMM-HMF-HOPS and VNMM-HMF-RES, respectively. In Figure 6c, the advantage of VNMM-WRE is more obvious in LAR/CR. It is approximately 31.8% and 68.3% higher than VNMM-HMF-HOPS and VNMM-HMF-RES, respectively. In VNMM-HMF-WRE, the resource and topology indicators are both used to reduce substrate resource consumption in VN reconfiguration. Also, WRE is introduced to calculate the candidate substrate node importance.  topology indicators are both used to reduce substrate resource consumption in VN reconfiguration. Also, WRE is introduced to calculate the candidate substrate node importance.   As can be seen from Figure 8a-c, increasing the lifetime of failure can reduce the available substrate resources, which not only decreases the AR, but also results in a reduction in resources used for VN reconfiguration. Therefore, with the increase of the lifetime of failure, the AR, NOR, and LAR/CR are all reduced significantly.

Conclusions and Future Work
In this paper, we have addressed the problem of recovering faulty VNs affected by hybrid multiple failures in SN. In this regard, we have formulated the VN reconfiguration to an ILP model to maximize the LAR/CR. We have proposed an efficient heuristic policy called DVNRM-HMF-WRE to solve this ILP model. In DVNRM-HMF-WRE, the WRE method is introduced into FVNRRM-WRE and VNMM-WRE. In FVNRRM-WRE, the WRE method is used to rank the multiple faulty VNs. In VNMM-WRE, candidate substrate node are selected based on WRE method which takes resource and topology indicators into consideration. Also, a failure handling method called HMFRA is proposed, which handles virtual node failure and VN link failure I first. At last, five experiments are designed. The first experiment verifies that the proposed DVNRM-HMF-WRE has excellent performance than other typical VN reconfiguration methods. The next three experiments assess that our proposed faulty VN ranking method, failure handling method, and VN node migration method As can be seen from Figure 8a-c, increasing the lifetime of failure can reduce the available substrate resources, which not only decreases the AR, but also results in a reduction in resources used for VN reconfiguration. Therefore, with the increase of the lifetime of failure, the AR, NOR, and LAR/CR are all reduced significantly.

Conclusions and Future Work
In this paper, we have addressed the problem of recovering faulty VNs affected by hybrid multiple failures in SN. In this regard, we have formulated the VN reconfiguration to an ILP model to maximize the LAR/CR. We have proposed an efficient heuristic policy called DVNRM-HMF-WRE to solve this ILP model. In DVNRM-HMF-WRE, the WRE method is introduced into FVNRRM-WRE and VNMM-WRE. In FVNRRM-WRE, the WRE method is used to rank the multiple faulty VNs. In VNMM-WRE, candidate substrate node are selected based on WRE method which takes resource and topology indicators into consideration. Also, a failure handling method called HMFRA is proposed, which handles virtual node failure and VN link failure I first. At last, five experiments are designed. The first experiment verifies that the proposed DVNRM-HMF-WRE has excellent performance than other typical VN reconfiguration methods. The next three experiments assess that our proposed faulty VN ranking method, failure handling method, and VN node migration method perform better than other corresponding methods. The last experiment sets two different scenarios to evaluate the performance of DVNRM-HMF-WRE in different arrival rates and lifetimes of failures. Evaluation results show that decreasing the arrival rate and the lifetime of failure can improve the performance of VN reconfiguration.
In future research, we will introduce the privacy and energy consumption into VN reconfiguration. Without adequate protection, users from a VN might be able to gain unauthorized access to data being transmitted through other VNs, violating the privacy of the entities that own those networks. Hence, privacy-oriented VNE and preserving privacy with VN stacks gradually get more and more attention. Also, energy consumption has become another hot topic in NV. With the increase of communication traffic every year, some energy models have been proposed to minimize the energy consumption in NV, such as the load-dependent power consumption model in VNE, the energy-efficient resource management for real-time service function chain in fog-supported SDN and network-aware energy optimization model for VN. Therefore, the ILP formulation and the heuristic algorithm could be extended to consider the privacy and energy consumption in future work.
Supplementary Materials: supplementary materials are available online at http://www.mdpi.com/1099-4300/ 20/9/711/s1. Author Contributions: Y.S. performed initially the theoretical derivation, designed the experiments and software code, and wrote the draft of this paper in Chinese; X.M. proposed the original idea, checked the initial derivation, revised radically the paper, and rewrote the paper in English; Q.K. made a major contribution to the revision of this paper as for the reviews' comments. X.H. analyzed the experimental data.

Conflicts of Interest:
The authors declare no conflict of interest.

Abbreviations
The following abbreviations are used in this manuscript: