Open Access
This article is

- freely available
- re-usable

*Entropy*
**2018**,
*20*(9),
711;
https://doi.org/10.3390/e20090711

Article

Dynamic Virtual Network Reconfiguration Method for Hybrid Multiple Failures Based on Weighted Relative Entropy

College of Information and Navigation, Air Force Engineering University, Xi’an 710077, China

^{*}

Author to whom correspondence should be addressed.

Received: 9 August 2018 / Accepted: 13 September 2018 / Published: 15 September 2018

## Abstract

**:**

Network virtualization can offer more flexibility and better manageability for next generation Internet. With the increasing deployments of virtual networks in military and commercial networks, a major challenge is to ensure virtual network survivability against hybrid multiple failures. In this paper, we study the problem of recovering virtual networks affected by hybrid multiple failures in substrate networks and provide an integer linear programming formulation to solve it. We propose a heuristic algorithm to tackle the complexity of the integer linear programming formulation, which includes a faulty virtual network reconfiguration ranking method based on weighted relative entropy, a hybrid multiple failures ranking algorithm, and a virtual node migration method based on weighted relative entropy. In the faulty virtual network reconfiguration ranking method based on weighted relative entropy and virtual node migration method based on weighted relative entropy, multiple ranking indicators are combined in a suitable way based on weighted relative entropy. In the hybrid multiple failures ranking algorithm, the virtual node and its connective virtual links are re-embedded, firstly. Evaluation results show that our heuristic method not only has the best acceptance ratio and normal operation ratio, but also achieves the highest long-term average revenue to cost ratio compared with other virtual network reconfiguration methods.

Keywords:

virtual network; weighted relative entropy; reconfiguration; hybrid multiple failures; survivability## 1. Introduction

Network virtualization (NV) allows multiple heterogeneous virtual networks (VNs) to be embedded onto the shared substrate network (SN), providing users with a variety of network services which has become one of the most promising technologies for future Internet [1,2,3]. It can decouple the network infrastructure and network services, and allow multiple heterogeneous VNs to share the SN resources through abstraction, distribution and isolation mechanism. In recent years, NV has been widely used in various fields, such as fog-supported software defined network (SDN) [4] and virtualized networked data centers [5]. One of the most challenging problems raised in NV is the virtual network embedding (VNE) problem, which has been known as the non-deterministic polynomial hard (NP-hard) problem [6].

In recent years, the SN failure events occur frequently. SN failures degrade the service performance and reliability of VNs [7]. SN failures include node failure and link failure. In network virtualization, multiple VNs are embedded onto the same SN and share the common substrate resources. Hence, substrate node failure causes virtual node and its adjacent virtual link failure. Substrate link failure causes virtual link failure. Also, there are single node/link failures and multiple node/link failures in SN which will cause complex and multiple virtual network (VN) failures [8]. Therefore, adopting suitable survivability strategies to against the increasingly complex network failures has become one of the main challenges in VN survivability.

A number of mechanisms have been proposed to improve the VN survivability [9]. These mechanisms can be broadly classified into two categories: backup mechanism [10] and reconfiguration mechanism [11]. The backup mechanism can also be classified into dedicated backup [12] and shared backup [13]. In shared backup mechanism, the backup resources can be shared with different VNs. In the dedicated case, the backup resources only can be used for dedicated VN. However, preallocating a backup resource is extremely expensive.

Instead, the reconfiguration mechanism can re-embed the faulty part of a VN without huge backup resource consumption. In reconfiguration mechanisms, backup resources are not preallocated, and the faulty parts of VNs are re-embedded after SN failures [14]. For instance, a topology awareness VN reconfiguration algorithm is proposed in [15]. The virtual node to be migrated is mapped onto the nearest mapped substrate node which is the neighbor mapping node of the migration virtual node. This algorithm takes topology indicator into consideration and improves the resource utilization. However, resource indicator is not considered in its node re-embedding strategy. To survive link failures, a novel recovery approach to restore VN without reserving backup resources prior to embedding is proposed in [16]. It not only considers the end-delay and delay variation requirements, but also takes actions to find a constrained shortest path between the nodes utilized by the failed VNs. The resource utilization is improved. However, this algorithm cannot deal with the problem of recovering VNs from node failures. As can be seen from above two algorithms, they cannot deal with the hybrid failures which include node failures and link failures.

To survive hybrid failures in SN, a heuristic survivable VNE based on node migration and link remapping is proposed in [17]. In VNE, the artificial bee colony algorithm is proposed to achieve an optimal solution. If the substrate node fails, the failed node is migrated to a normal node based on the greed rules, and the affected links are remapped based on the shortest path algorithm. Then, the shortest path is used to provide the best bandwidth principle to remap the affected links that connect with the failed node. This algorithm can deal with hybrid failures. However, the greed rules only take resource indicator into consideration and their re-embedding performance is limited. Also, they do not consider the hybrid failures in multiple faulty VNs.

To solve the problem of recovering a batch of VNs affected by a substrate node failure, a recovery approach for maximizing recovery and minimizing the cost of recovery is designed in [18]. An integer linear programming (ILP) formulation of this recovery scheme is provided, and a fast and scalable heuristic algorithm is also proposed to tackle the computational complexity of the ILP solution. It can recover a batch of VNs affected by a substrate node failure efficiently. However, it sorts the faulty VNs and virtual nodes based on the adjacent bandwidth resource, and it ignores the topology indicators. A generalized recovery approach that can achieve customized objectives is designed, and the corresponding ILP formulation is provided in [19]. Then, a fast and scalable heuristic algorithm to tackle the computational complexity of the ILP solution is proposed, and it is demonstrated that this heuristic algorithm has good recovery performance. However, it focuses on the problem of recovering from a node failure in VNE and ranks faulty VNs based on service level agreements. It cannot deal with the problem of recovering a batch of VNs affected by hybrid failures which include node and link failures. Also, in different recovery objects, recovering the faulty VNs based on service level agreements is not suitable.

As can be seen from the current research, the VN reconfiguration mechanisms focus on taking effective measures for network recovery after failures without preallocating resources. Although numerous efforts have been devoted, there are four obvious defects in existing VN reconfiguration algorithms. At first, VN reconfiguration for different types of failures in one VN and VN reconfiguration for single type of failures in multiple VNs are both studied. However, the VN reconfiguration for multiple types of failures in multiple VNs is not studied. Secondly, in reconfiguration of multiple faulty VNs, a single indicator is selected in most faulty VN ranking strategies. In different recovery objects, these strategies are not always suitable. Thirdly, a single resource indicator or topology indicator is taken into consideration in VN re-embedding, and the performance is limited. At last, the failures are not classified into certain types, and the recovery order of different types of failures is not researched.

In this paper, an ILP formulation of VN reconfiguration is provided, firstly. Then, the heuristic algorithm, called dynamic virtual network reconfiguration method for hybrid multiple failures based on weighted relative entropy (DVNRM-HMF-WRE), is proposed to solve the ILP formulation. In DVNRM-HMF-WRE, the faulty VN reconfiguration ranking method based on weighted relative entropy (FVNRRM-WRE), hybrid multiple failures ranking algorithm (HMFRA), and virtual node migration method based on weighted relative entropy (VNMM-WRE), are proposed. At last, five comparative experiments are set to demonstrate the performance of DVNRM-HMF-WRE.

The main contributions of this paper can be summarized as follows:

- (1)
- We provide an ILP formulation of VN reconfiguration and propose the DVNRM-HMF-WRE to solve it. VNE is already intractable, and the combinatorial number of sequences of VNs in batch re-embedding further increases the complexity. Therefore, we only re-embed the faulty nodes and links without disrupting their unaffected parts in VN.
- (2)
- We introduce a weighted relative entropy (WRE) method into FVNRRM-WRE and VNMM-WRE. In FVNRRM-WRE, we rank the faulty VNs with the help of multiple indicators, and combine them based on WRE. In VNMM-WRE, we use the WRE method to change the coefficients of node resource, and topology indicator to select the suitable candidate substrate node to reduce the resource consumption.
- (3)
- We propose the HMFRA to handle the hybrid multiple failures. In HMFRA, faulty virtual node and its connective virtual links are handled first, which are the hardest the recover. HMFRA can make full use of the limited resources and improve the recovery performance.
- (4)
- Five different simulation scenarios are conducted to validate the performance of DVNRM-HMF-WRE. The performance of DVNRM-HMF-WRE is compared with other algorithms in the first simulation experiment. Next, we simulate the impact of FVNRRM-WRE, HMFRA, and VNMM-WRE on the performance of DVNRM-HMF-WRE. At last, two factors are set to evaluate their influences on DVNRM-HMF-WRE.

The rest of this paper is organized as follows. Section 2 presents the models and evaluation indicators. In Section 3, we propose the ILP of VN reconfiguration. The DVNRM-HMF-WRE is given, and its details are shown in Section 4. In Section 5, we evaluate the proposed algorithm through extensive simulations and experiments. We conclude this paper with a summary and areas for future exploration in Section 6.

## 2. Models and Evaluation Indicators

In this section, we describe the models and evaluation indicators. Here, Table 1 summarizes the main notations used in this paper.

#### 2.1. Network Model

#### 2.1.1. Substrate Network

The SN is modeled as a weighted undirected graph

**G**_{S}= (N_{S}, E_{S}), in which the substrate node set and substrate link set are represented by N_{S}and E_{S}, respectively. In substrate nodes, the available CPU resource of substrate node n_{s}is denoted by cpu(n_{s}), and the location attribute of substrate node n_{s}is denoted by loc(n_{s}). Similarly, bw(e_{s}) is taken to denote the available bandwidth resource of substrate link e_{s}. Also, N_{Sf}represents faulty substrate node set and E_{Sf}represents the faulty substrate link set.#### 2.1.2. Virtual Network

Similar to SN, the VN can be modeled as a weighted undirected graph

**G**_{V}= (N_{V}, E_{V}). N_{V}represents virtual node set and E_{V}represents the virtual link set. In virtual nodes, cpu(n_{v}) denotes the required CPU resource of virtual node n_{v}. In virtual links, bw(e_{v}) is taken as the basic attribute to denote the required bandwidth resource of virtual link e_{v}. T_{V}is used to denote the VN lifetime.**G**_{Vf}represents the faulty VN set. Also, N_{Vf}represents faulty virtual node set and E_{Vf}represents the faulty virtual link set.#### 2.2. Failure Model

#### 2.2.1. Multiple Failures

The SN failures are denoted by a series of failure events F

_{i,j}, I = 1,2; j = 1, 2,...,n. F_{1,j}denotes the substrate node failure and F_{1,j}∈N_{Sf}. F_{2,j}denotes the substrate link failure and F_{2,j}∈E_{Sf}, j is the failure number. t_{s}(F_{i,j}) and t_{e}(F_{i,j}) denote the starting and ending time of F_{i,j}, and t_{s}(F_{i,j}) < t_{e}(F_{i,j}). When ∃j1 < j2, if t_{s}(F_{i,j}_{1}) < t_{s}(F_{i,j}_{2}) < t_{e}(F_{i,j}_{1}), multiple failures happen in SN. Multiple failures contain multiple node failures and multiple link failures.#### 2.2.2. Failure Types

VN failures caused by SN failures can be divided into virtual node failures and virtual link failures. Also, virtual link failures can be divided into three categories. If virtual link takes faulty substrate node as the source or destination node, it is called VN link failure I. The virtual link which passes through the faulty substrate node but does not take it as the source or destination node, is called VN link failure II. The independent virtual link failure caused by independent substrate link failure is called VN link failure III.

#### 2.2.3. Hybrid Multiple Failures

If multiple failures occur at the same time and belong to multiple types, they are called hybrid multiple failures.

The process of VNE and VN reconfiguration are shown in Figure 1. Figure 1a shows two VNs, and Figure 1b illustrates an example for VNE problem. In VN1, the node embedding results are {a→A, b→B, c→C}, and link embedding results are {(a,b)→(A,B), (a,c)→(A,C)}, in which both the CPU and bandwidth requirements are satisfied. In VN2, the node embedding results are {d→H, e→G, f→I} and link embedding results are {(d,e)→(H,G), (d,f)→(H,I), (e,f) →(G,D,B,A,C,E,I)}.

Figure 1c,d illustrate the examples for VN reconfiguration. In Figure 1c, the substrate node A is faulty, which causes the connected substrate link (A,B) and (A,C) failure. The faulty virtual link (a,b) and (a,c) belongs to VN link failure I. Also, virtual link (e,f) in VN2 is faulty, which belongs to VN link failure II. At this time, virtual node a is re-embedded onto substrate node D. Virtual link (a,b) and (a,c) are re-embedded onto substrate link (D,B) and (D,F,E,C), respectively. After that, VN1 returns to normal. Virtual link (e,f) is re-embedded onto substrate link (G,D,F,E,I), then VN2 returns to normal.

In Figure 1d, substrate link (B,D) is faulty, which causes the virtual link (e,f) failure and this belongs to VN link failure III. After re-embedding virtual link (e,f) onto substrate link (G,D,F,E,I), VN2 returns to normal.

#### 2.3. Evaluation Indicator

#### 2.3.1. Acceptance Ratio

The acceptance ratio (AR) is denoted by the number of VN requests which are embedded successfully, divided by the total number of VN requests. It is shown in Formula (1).
where δ is infinitely close to 0. $\left|VN(t)\right|$ is the total number of VN requests at time t and $\left|V{N}_{\mathrm{map}}(t)\right|$ is the number of VNs which are embedded successfully at time t.

$$r={\displaystyle \underset{T\to \infty}{\mathrm{lim}}}\frac{{\displaystyle \sum _{t=0}^{T}\left|V{N}_{\mathrm{map}}(t)\right|}}{{\displaystyle \sum _{t=0}^{T}\left|VN(t)\right|+\delta}},$$

#### 2.3.2. Revenue to Cost Ratio

For VN request
where α and β are weighting coefficients to balance CPU and bandwidth resources. In this paper, we set α = β = 1. Also, hops(e

**G**_{V}= (N_{V}, E_{V}), we denote revenue R(**G**_{V}, t) and cost C(**G**_{V}, t) as
$$R({\mathit{G}}_{\mathrm{V}},t)=\alpha {\displaystyle \sum _{{n}_{\mathrm{v}}\in {N}_{\mathrm{V}}}cpu({n}_{\mathrm{v}})}+{\displaystyle \sum _{{e}_{\mathrm{v}}\in {E}_{\mathrm{V}}}bw({e}_{\mathrm{v}})},$$

$$C({\mathit{G}}_{\mathrm{V}},t)=\beta {\displaystyle \sum _{{n}_{\mathrm{v}}\in {N}_{\mathrm{V}}}cpu({n}_{\mathrm{v}})}+{\displaystyle \sum _{{e}_{\mathrm{v}}\in {E}_{\mathrm{V}}}hops({e}_{\mathrm{v}})bw({e}_{\mathrm{v}})},$$

_{v}) is the hop counts in substrate link corresponding to virtual link e_{v}.At time t
where t

_{i}, if**G**_{V}is faulty and can be re-embedded successfully, the revenue and cost will not change. Otherwise, the revenue and cost of**G**_{V}are set to zero, and it also causes the penalty cost PCost which can be defined as
$$PCost({\mathit{G}}_{\mathrm{V}},t)=\mu \cdot ({t}_{j}-{t}_{i})\cdot R({\mathit{G}}_{\mathrm{V}},t),$$

_{j}is terminal time of**G**_{V}and μ is the penalty coefficient. In this paper, μ = 2.Long-term average revenue to cost ratio (LAR/CR) can be defined as
where VN

$$\xi ={\displaystyle \underset{T\to \infty}{\mathrm{lim}}}\frac{{\displaystyle \sum _{t=0}^{T}{\displaystyle \sum _{{\mathit{G}}_{\mathrm{V}}\subset V{N}_{\mathrm{map}}(t)}R\left({\mathit{G}}_{\mathrm{V}},t\right)}-{\displaystyle \sum _{t=0}^{T}{\displaystyle \sum _{{\mathit{G}}_{\mathrm{V}}\subset V{N}_{\mathrm{fr}}(t)}PCost\left({\mathit{G}}_{\mathrm{V}},t\right)}}}}{{\displaystyle \sum _{t=0}^{T}{\displaystyle \sum _{{\mathit{G}}_{\mathrm{V}}\subset V{N}_{\mathrm{map}}(t)}C\left({\mathit{G}}_{\mathrm{V}},t\right)}}},$$

_{fr}(t) is the set of faulty VNs which are re-embedded unsuccessfully. VN_{map}(t) is the VNs which are embedded successfully at time t.#### 2.3.3. Normal Operation Ratio

At time t, |VN

_{fr}(t)| denotes the number of faulty VNs which are unsuccessfully re-embedded. |VN(t)| is the total number of embedded VN requests. The normal operation ratio (NOR) can be defined as
$$\eta (t)=\frac{\left|VN(t)\right|-\left|V{N}_{\mathrm{fr}}(t)\right|}{\left|VN(t)\right|}.$$

## 3. ILP of VN Reconfiguration

In this section, we formulate the ILP of VN reconfiguration. The objective function and constraints can be expressed as follows.

#### 3.1. Objective Function

$$\mathrm{max}\{{\displaystyle \underset{T\to \infty}{\mathrm{lim}}}\frac{{\displaystyle \sum _{t=0}^{T}{\displaystyle \sum _{{\mathit{G}}_{\mathrm{V}}\subset V{N}_{\mathrm{map}}(t)}R\left({\mathit{G}}_{\mathrm{V}},t\right)}-{\displaystyle \sum _{t=0}^{T}{\displaystyle \sum _{{\mathit{G}}_{\mathrm{V}}\subset V{N}_{\mathrm{fr}}(t)}PCost\left({\mathit{G}}_{\mathrm{V}},t\right)}}}}{{\displaystyle \sum _{t=0}^{T}{\displaystyle \sum _{{\mathit{G}}_{\mathrm{V}}\subset V{N}_{\mathrm{map}}(t)}C\left({\mathit{G}}_{\mathrm{V}},t\right)}}}\}.$$

In this paper, our object is to get the maximum long-term average revenue to cost ratio. The corresponding variables in Formula (7) can be seen from Formulas (2)–(5) or Table 1.

#### 3.2. Constraints

$$\begin{array}{c}\forall {n}_{\mathrm{v}}\in {N}_{\mathrm{V}},\text{}\forall {n}_{\mathrm{s}}\in {N}_{\mathrm{S}},\\ x({n}_{\mathrm{v}},{n}_{\mathrm{s}})=\{\begin{array}{l}1iff{n}_{\mathrm{v}}\mathrm{is}\text{re-embedded}\mathrm{onto}{n}_{\mathrm{s}}\\ 0\mathrm{otherwise}\end{array}.\end{array}$$

$$\begin{array}{c}\forall {e}_{up}\in {E}_{\mathrm{V}},\text{}\forall {e}_{ij}\in {E}_{\mathrm{S}},\\ x({e}_{up},{e}_{ij})=\{\begin{array}{l}1iff{e}_{up}\mathrm{is}\text{re-embedded}\mathrm{onto}{e}_{ij}\\ 0\mathrm{otherwise}\end{array}.\end{array}$$

$$\begin{array}{c}\forall {n}_{\mathrm{v}}\in {N}_{\mathrm{V}},\text{}\forall {n}_{\mathrm{s}}\in {N}_{\mathrm{S}},\\ s({n}_{\mathrm{v}})=\{\begin{array}{l}1iff{n}_{\mathrm{v}}\mathrm{is}\mathrm{faulty}\\ 0\mathrm{otherwise}\end{array},\text{}s({n}_{\mathrm{s}})=\{\begin{array}{l}1iff{n}_{\mathrm{s}}\mathrm{is}\mathrm{faulty}\\ 0\mathrm{otherwise}\end{array}.\end{array}$$

$$\forall {e}_{\mathrm{v}}\in {E}_{\mathrm{V}},\text{}\forall {e}_{\mathrm{s}}\in {E}_{\mathrm{S}},\phantom{\rule{0ex}{0ex}}s({e}_{\mathrm{v}})=\{\begin{array}{l}1iff{e}_{\mathrm{v}}\mathrm{is}\mathrm{faulty}\\ 0\mathrm{otherwise}\end{array},\text{}s({e}_{\mathrm{s}})=\{\begin{array}{l}1iff{e}_{\mathrm{s}}\mathrm{is}\mathrm{faulty}\\ 0\mathrm{otherwise}\end{array}.\phantom{\rule{0ex}{0ex}}s({\mathit{G}}_{\mathrm{V}})=\{\begin{array}{l}1iff{\mathit{G}}_{\mathrm{V}}\mathrm{is}\mathrm{faulty}\\ 0\mathrm{otherwise}\end{array},\text{}s({\mathit{G}}_{\mathrm{S}})=\{\begin{array}{l}1iff{\mathit{G}}_{\mathrm{S}}\mathrm{is}\mathrm{faulty}\\ 0\mathrm{otherwise}\end{array}.$$

In constraint (8), if virtual node n

_{v}is re-embedded onto substrate node n_{s}, x(n_{v}, n_{s}) = 1. Otherwise, x(n_{v}, n_{s}) = 0. In constraint (9), if virtual link e_{up}is re-embedded onto substrate link e_{ij}, x(e_{up}, e_{ij}) = 1. Otherwise, x(e_{up}, e_{ij}) = 0. In constraint (10), if virtual node n_{v}is faulty, s(n_{v}) = 1. Otherwise, s(n_{v}) = 0. Also, if substrate node n_{s}is faulty, s(n_{s}) = 1. Otherwise, s(n_{s}) = 0. In constraint (11), if virtual link e_{v}is faulty, s(e_{v}) = 1. Otherwise, s(e_{v}) = 0. If substrate link e_{s}is faulty, s(e_{s}) = 1. Otherwise, s(e_{s}) = 0. If virtual network**G**_{V}is faulty, s(**G**_{V}) = 1. Otherwise, s(**G**_{V}) = 0. If substrate network**G**_{S}is faulty, s(**G**_{S}) = 1. Otherwise, s(**G**_{S}) = 0.
$$\forall {n}_{\mathrm{v}}\in {N}_{\mathrm{V}},\text{}\forall {n}_{\mathrm{s}}\in {N}_{\mathrm{S}},\phantom{\rule{0ex}{0ex}}cpu({n}_{\mathrm{v}})\times x({n}_{\mathrm{v}},{n}_{\mathrm{s}})\le cpu({n}_{\mathrm{s}}),$$

$$\forall {e}_{up}\in {E}_{\mathrm{V}},\text{}\forall {e}_{ij}\in {E}_{\mathrm{S}},\phantom{\rule{0ex}{0ex}}{\displaystyle \sum _{\forall {e}_{up}\in {E}_{\mathrm{V}}}bw({e}_{up})\times x({e}_{up},{e}_{ij})}\le bw({e}_{ij}).$$

Constrains (12) and (13) are resource constraints, and they ensure that if virtual node n

_{v}and virtual link e_{up}are selected to re-embed, the candidate substrate node n_{s}and substrate link e_{ij}should have more resources than the virtual node n_{v}and virtual link e_{up}request, respectively.
$$x({n}_{\mathrm{v}},{n}_{\mathrm{s}})\times dis(loc({n}_{\mathrm{s}}),loc({n}_{\mathrm{v}}))\le D({n}_{\mathrm{s}}).$$

Constrain (14) is location constraint and dis(loc(n

_{s}), loc(n_{v})) denotes the distance between substrate node n_{s}and virtual node n_{v}.
$$\forall {n}_{\mathrm{v}}\in {N}_{\mathrm{V}},\text{}\forall {n}_{\mathrm{s}}\in {N}_{\mathrm{S}},\text{}\forall {e}_{up}\in {E}_{\mathrm{V}},\text{}\forall {e}_{ij}\in {E}_{\mathrm{S}},\phantom{\rule{0ex}{0ex}}\begin{array}{l}{\displaystyle \sum _{{e}_{ji}\in {E}_{\mathrm{S}}}x({e}_{up},{e}_{ji})}-{\displaystyle \sum _{{e}_{ij}\in {E}_{\mathrm{S}}}x({e}_{up},{e}_{ij})}\\ =\{\begin{array}{ll}1,& x({n}_{u},{n}_{j})=1\\ -1,& x({n}_{u},{n}_{j})=1\\ 0,& \mathrm{otherwise}\end{array}\end{array}$$

Constrain (15) is the connectivity constraint. It refers to the flow conservation constraint for routing one unit of net flow from virtual node n

_{u}to virtual node n_{p}.
$$\forall {n}_{\mathrm{s}}\in {N}_{\mathrm{S}},\text{}{\displaystyle \sum _{{n}_{\mathrm{v}}\in {N}_{\mathrm{V}}}x({n}_{\mathrm{v}},{n}_{\mathrm{s}})}\le 1\text{}$$

$$\forall {n}_{\mathrm{v}}\in {N}_{\mathrm{V}},\text{}{\displaystyle \sum _{{n}_{\mathrm{s}}\in {N}_{\mathrm{S}}}x({n}_{\mathrm{v}},{n}_{\mathrm{s}})}=1\text{}$$

Constrains (16) and (17) are re-embedding constraints. Constrain (16) ensures that all virtual nodes in same VN are re-embedded on different substrate nodes. Constrain (17) ensures that each virtual node is re-embedded on up to one substrate node.

$$\mathrm{if}\forall {n}_{\mathrm{v}}\in {N}_{\mathrm{V}}\mathrm{and}s({n}_{\mathrm{v}})=1,\text{}\mathrm{then}s({\mathit{G}}_{\mathrm{V}})=1\text{}$$

$$\mathrm{if}\forall {e}_{up}\in {E}_{\mathrm{V}}\mathrm{and}s({e}_{up})=1,\mathrm{then}s({\mathit{G}}_{\mathrm{V}})=1\text{}$$

$$\forall {n}_{i}\in {N}_{\mathrm{S}},\text{}\forall {n}_{j}\in {N}_{\mathrm{S}},\text{}\forall {e}_{ij}\in {E}_{\mathrm{S}},\phantom{\rule{0ex}{0ex}}\mathrm{if}s({n}_{i})=1,\mathrm{then}s({e}_{ij})=1\text{}$$

$$\forall {n}_{q}\in {N}_{\mathrm{Vf}},\text{}\forall {n}_{j}\in {N}_{\mathrm{Sf}},\text{}x({n}_{q},{n}_{j})=0\text{}$$

$$\forall {e}_{up}\in {E}_{\mathrm{Vf}},\text{}\forall {e}_{ij}\in {E}_{\mathrm{Sf}},\text{}x({e}_{up},{e}_{ij})=0\text{}$$

Constrains (18)–(22) are failure constraints. Constrains (18) and (19) ensure that if either the virtual node or virtual link fail in VN, the VN will also fail. Constrain (20) ensures that if one of the end-nodes in substrate link fails, the substrate link will fail. Constrain (21) ensures that virtual node and the corresponding substrate node re-embedded by it cannot fail at the same time. Constrain (22) ensures that virtual link and the corresponding substrate link re-embedded by it cannot fail at the same time.

This ILP is known to be NP-hard problem, and solving it is computationally intractable. Even though optimal results can be obtained by some exact algorithms or open source linear programming toolkit GLPK, they may incur exponential increasing running time. Consequently, they cannot be scaled to address large-scale VNs. Hence, most researchers solve the ILP problem of VN reconfiguration by proposing a corresponding heuristic algorithm which has short computational time and gets an approximate optimal solution. Therefore, we propose a novel heuristic algorithm called DVNRM-HMF-WRE to solve the ILP of VN reconfiguration.

## 4. DVNRM-HMF-WRE

#### 4.1. Problem Statement

The VN reconfiguration is a complex and dynamic process. There are many event statements at the same time. We analyze the possible event statements and provide the dynamic process of VN reconfiguration first. Then, three algorithms in DVNRM-HMF-WRE are proposed.

VN embeds and leaves dynamically. Many types of failures occur randomly in SN. At time t, there are four possible event statements:

- When the lifetime of VN ends, the corresponding SN resources are released.
- When the faulty component in SN is repaired, its resources also return to normal.
- When SN failure occurs, the resources of faulty component are set to zero and the VN embedded on it needs to be re-embedded. If multiple failures occur at the same time, the faulty VNs will be re-embedded one by one. When all faulty virtual components in one VN are re-embedded successfully, the VN reconfiguration is successful.
- When VN request arriving, VN begins to embed. If succeed, the corresponding resources are used.

When multiple SN failures occur, the process of DVNRM-HMF-WRE is shown in Figure 2.

As can be seen that, from Figure 2, when failure occurs, the resources of faulty components are set to zero. If there are some VNs embedded onto the faulty components, the resources used by faulty components in VNs are released. Then, the DVNRM-HMF-WRE is used to re-embed the faulty VNs. In DVNRM-HMF-WRE, the FVNRRM-WRE is used to rank multiple faulty VNs to decide which faulty VN should be re-embedded first. The HMFRA and VNMM-WRE are used together to recover the hybrid multiple failures in one VN. The HMFRA is proposed to rank hybrid multiple faulty nodes/links in one VN, and the VNMM-WRE is provided to improve the efficiency of virtual node migration. If the VN reconfiguration is successful, the resource statement of SN I updated, otherwise, the resources used by normal components in faulty VN are released. If all VN reconfigurations are over, the DVNRM-HMF-WRE is stopped.

#### 4.2. FVNRRM-WRE

When multiple failures occur at the same time, and substrate resources are limited, the reconfiguration sequence of faulty VNs is especially important. In faulty VN ranking strategies, ranking indicators and ranking method are very important. Among them, faulty component number, VN revenue, and remaining lifetime of the faulty VN all have obvious impact on the reconfiguration performance. They are selected to rank the faulty VNs. When ranking faulty VNs, a fixed formula is usually used, and it cannot adapt to the changing environment. Therefore, the WRE method is introduced in this paper.

The WRE method is a classical ranking method which can adjust the indicator coefficients to adapt the changing environment. It is improved by the classical technique for order preference by similarity to an ideal solution (TOPSIS) [20]. In TOPSIS, the generalized distance is unable to distinguish the point in perpendicular bisector between positive and negative ideal solution [21]. Therefore, the relative entropy is used, instead, to the generalized distance.

For system A and B in state A

_{i}and B_{i}(i = 1, 2,…,N), their relative entropy, C, can be defined by
$$C={\displaystyle \sum _{i=1}^{N}[{A}_{i}\mathrm{log}\frac{{A}_{i}}{{B}_{i}}+(1-{A}_{i})\mathrm{log}\frac{1-{A}_{i}}{1-{B}_{i}}]}.$$

There are N faulty VNs, and each faulty VN has M evaluation indicators. The j-th indicator coefficient of the i-th node is denoted as X

_{ij}(i = 1, 2, …, N; j = 1, 2, …, M), and all coefficients of faulty VNs constitute a decision matrix**X**which is denoted as
$$\mathit{X}=\left(\begin{array}{cccc}{x}_{11}& {x}_{12}& \cdots & {x}_{1M}\\ {x}_{21}& {x}_{22}& \cdots & {x}_{2M}\\ \vdots & \vdots & \ddots & \vdots \\ {x}_{N1}& {x}_{N2}& \cdots & {x}_{NM}\end{array}\right).$$

Also, the coefficient is normalized to make a fair comparison between different indicators.

In different environments, each indicator has a different importance. The weighting coefficient of the j-th indicator is expressed as ${\omega}_{j}$($j=1,2,\dots ,M$, $\sum {\omega}_{j}}=1$), and the weighted normalized decision matrix is denoted by

$$\mathit{Y}=\mathit{X}\mathit{\omega}=\left(\begin{array}{cccc}{x}_{11}{\omega}_{1}& {x}_{12}{\omega}_{2}& \cdots & {x}_{1M}{\omega}_{M}\\ {x}_{21}{\omega}_{1}& {x}_{22}{\omega}_{2}& \cdots & {x}_{2M}{\omega}_{M}\\ \vdots & \vdots & \ddots & \vdots \\ {x}_{N1}{\omega}_{1}& {x}_{N2}{\omega}_{2}& \cdots & {x}_{NM}{\omega}_{M}\end{array}\right).$$

The positive and negative ideal solutions A

^{+}and A^{−}are determined as
$${A}^{+}=\{\mathrm{max}({y}_{i1},{y}_{i2},\dots ,{y}_{iM})\}=\{{y}_{1}^{\mathrm{max}},{y}_{2}^{\mathrm{max}},\dots ,{y}_{M}^{\mathrm{max}}\},$$

$${A}^{-}=\{\mathrm{min}({y}_{i1},{y}_{i2},\dots ,{y}_{iM})\}=\{{y}_{1}^{\mathrm{min}},{y}_{2}^{\mathrm{min}},\dots ,{y}_{M}^{\mathrm{min}}\}.$$

The relative entropies of each solution to positive and negative ideal solution are calculated as

$${C}_{i}^{+}={\displaystyle \sum _{j=1}^{M}\left[{y}_{j}^{\mathrm{max}}\mathrm{log}\frac{{y}_{j}^{\mathrm{max}}}{{y}_{ij}}+(1-{y}_{j}^{\mathrm{max}})\mathrm{log}\frac{1-{y}_{j}^{\mathrm{max}}}{1-{y}_{ij}}\right]},$$

$${C}_{i}^{-}={\displaystyle \sum _{j=1}^{M}\left[{y}_{j}^{\mathrm{min}}\mathrm{log}\frac{{y}_{j}^{\mathrm{min}}}{{y}_{ij}}+(1-{y}_{j}^{\mathrm{min}})\mathrm{log}\frac{1-{y}_{j}^{\mathrm{min}}}{1-{y}_{ij}}\right]}.$$

The similarity between each solution and the ideal one is calculated as

$${Z}_{i}=\frac{{C}_{i}^{-}}{{C}_{i}^{-}+{C}_{i}^{+}},0\le {Z}_{i}\le 1.$$

The details of FVNRRM-WRE method are described in Algorithm 1.

Algorithm 1. The FVNRRM-WRE |

Input: X, weighting coefficients |

Output: Z_{i} |

1. Bring X and the weighting coefficient of each indicator into Formula (25) to construct the weighted normalized matrix Y. |

2. Calculate the positive and negative ideal scheme A^{+} and A^{−}. |

3. Calculate the positive and negative ideal solution C^{+} and C^{−}. |

4. Calculate Z_{i} and get the importance degrees of faulty VNs. |

In Algorithm 1,

**X**is the multiple indicators decision matrix which includes three indicators, called faulty components number, VN revenue, and remaining lifetime of the faulty VN. To calculate Z_{i}, which is the importance degree of faulty VN_{i}, the WRE method is widely used based on Formulas (26)–(30). After getting the importance degrees of faulty VNs, rank the faulty VNs in order of importance degree, from large to small, which can re-embed the important VN first.#### 4.3. HMFRA

In a faulty VN, there are multiple failures which include node failures and link failures. Recovery sequence is important in VN reconfiguration, and the HMFRA is proposed to recover different types of failures in a suitable manner. The details of HMFRA are shown in Algorithm 2.

Algorithm 2. The HMFRA |

Input: G_{S}, G_{V}, the set of faulty node, the set of VN link failure I, the set of VN link failure II, the set of VN link failure III |

Output: G_{S}, G_{V} |

1. if the faulty node set is not empty |

2. Migrate the virtual nodes using VNMM-WRE and re-embed all their connective links (VN |

link failures I) using k-shortest path algorithm. |

3. if all virtual nodes are migrated and all their adjacent links are re-embedded successfully |

4. Update G_{S} and G_{V} |

5. if the set of VN link failure II is not empty |

6. Re-embed all VN link failures II |

7. if all VN link failures II are re-embedded successfully |

8. Update G_{S} and G_{V} |

9. if the set of VN link failure III is not empty |

10. Re-embed all of them |

11. if all VN link failures III are re-embedded successfully |

12. Update G_{S} and G_{V} |

13. Return VN_RE-EMBED_SUCCESS |

14. else release the resources in G_{S} which are used by the normal parts |

15. Update G_{S} |

16. Return VN_RE-EMBED_FAILURE |

17. end if |

18. else update G_{S} and G_{V} |

19. Return VN_RE-EMBED_SUCCESS |

20. end if |

21. else release the resources in G_{S} which are used by the normal parts |

22. Update G_{S} |

23. Return VN_RE-EMBED_ FAILURE |

24. end if |

25. else update G_{S} and G_{V} |

26. Return VN_RE-EMBED_SUCCESS |

27. end if |

28. else release the resources in G_{S} which are used by the normal parts |

29. Update G_{S} |

30. Return VN_RE-EMBED_ FAILURE |

31. end if |

32. else |

33. if the set of VN link failure III is not empty |

34. Re-embed all of them |

35. if all VN link failures III are re-embedded successfully |

36. Update G_{S} and G_{V} |

37. Return VN_RE-EMBED_SUCCESS |

38. else release the resources in G_{S} which are caused by the normal parts |

39. Update G_{S} |

40. Return VN_RE-EMBED_ FAILURE |

41. end if |

42. end if |

43. end if |

As can be seen from Algorithm 2:

- Faulty virtual node and its connective links (VN link failures I) are recovered first. If substrate node is faulty, virtual node and its connective virtual links both need to be re-embedded. They will consume huge substrate node and link resources. They are the hardest to re-embed. If this succeeds, go on re-embedding virtual links in the set of VN link failure II. Otherwise, VN reconfiguration has failed.
- If the VN link failures II are re-embedded successfully, then judge if there are VN link failures III. If there are VN link failures III, re-embed them. If this succeeds, VN reconfiguration is successful.
- If there is no VN link failure III after re-embedding the VN link failure II successfully, the faulty VN returns to normal. If the VN link failure II fails to re-embed, the faulty VN reconfiguration has failed.
- If there is only VN link failure III, re-embed these faulty virtual links. If this succeeds, VN reconfiguration is successful.

#### 4.4. VNMM-WRE

In HMFRA, the method of faulty virtual node migration is called VNMM-WRE, which is proposed in this section and the method of virtual link re-embedding is k-shortest path algorithm [22].

Similar to FVNRRM-WRE, the WRE method is also introduced into VNMM-WRE. In virtual node ranking, the importance degree of each virtual node Z(n

_{v}) is calculated using WRE method, which selects node CPU resource and node adjacent link bandwidth resource as the indicators. In substrate node ranking, the importance degree of each substrate node Z(n_{s}) is calculated using WRE method which selects node CPU resource, node adjacent link bandwidth resource and the reciprocal of hop counts between candidate substrate node and substrate node which is embedded by the neighboring virtual node of the migrated virtual node as the indicators [23].The details of VNMM-WRE are shown in Algorithm 3

Algorithm 3. The VNMM-WRE |

Input: Faulty virtual nodes which need to migrate |

Output: NodeMigraingList |

1. for each virtual node n_{v}∈N_{V} |

2. Calculate the Z(n_{v}) |

3. end for |

4. Rank virtual nodes in order of Z(n_{v}) from large to small |

5. Save virtual node ranking order into VirtualNodeList |

6. for each virtual node in VirtualNodeList |

7. Select the candidate substrate node set Can(n_{vi}) which satisfies the resource constraints |

8. Remove the embedded substrate nodes from Can(n_{vi}) |

9. if Can(n_{vi}) is empty |

10. Return NODE_ MIGRATE_FAILURE |

11. else |

12. for each candidate node n_{s} in Can(n_{vi}) |

13. Calculate the Z(n_{s}) of each substrate node in Can(n_{vi}) |

14. end for |

15. for each virtual node in Can(n_{vi}) |

16. Re-embed n_{v} to n_{s} which has the largest Z(n_{s}) value |

17. Re-embed the virtual links which connect the virtual node |

18. if all virtual links are re-embedded successfully |

19. Save the substrate node n_{s} into NodeMigratingList |

20. break |

21. else |

22. Remove the virtual node n_{v} from Can(n_{vi}) |

23. end if |

24. end for |

25. Update G_{S} and G_{V} |

26. Return NODE_MIGRATE_SUCCESS |

27. end if |

28. end for |

In Algorithm 3:

- Lines 1–5: calculate the Z(n
_{v}) of each faulty virtual node and rank them based on Z(n_{v}). Then save the ranking results into VirtualNodeList. - Lines 6–14: select the candidate substrate nodes and calculate their Z(n
_{s}). - Lines 15–28: re-embed the virtual node and its adjacent virtual links.

In summary, with the help of FVNRRM-WRE, HMFRA, and VNMM-WRE, the faulty VNs caused by hybrid multiple failures can be recovered.

#### 4.5. COMPLEXITY ANALYSIS

The DVNRM-HMF-WRE includes FVNRRM-WRE, HMFRA, and VNMM-WRE. In FVNRRM-WRE, all faulty VNs are ranked based on the WRE method. Its complexity is $O(\left|{N}_{\mathrm{FN}}\right|)$, in which $\left|{N}_{\mathrm{FN}}\right|$ is the total number of faulty VNs. The HMFRA and VNMM-WRE are used, together, to recover the hybrid multiple failures in VNs. In the recovery of faulty VNs, the complexity of virtual node re-embedding is $O(\left|{N}_{\mathrm{V}}\right|{\left|{N}_{\mathrm{S}}\right|}^{2})$. $\left|{N}_{\mathrm{V}}\right|$ is the total number of faulty virtual nodes in all faulty VNs and $\left|{N}_{\mathrm{S}}\right|$ is the total number of substrate nodes. The virtual link re-embedding algorithm is k-shortest path algorithm and its complexity is $O(k(\left|{N}_{\mathrm{V}}\right|+\left|{E}_{\mathrm{V}}\right|)(\left|{E}_{\mathrm{S}}\right|+\left|{N}_{\mathrm{S}}\right|\mathrm{lg}\left|{N}_{\mathrm{S}}\right|))$. $\left|{E}_{\mathrm{V}}\right|$ represents the total number of faulty virtual links in all faulty VNs and $\left|{E}_{\mathrm{S}}\right|$ represents the total number of substrate links. Therefore, the total complexity of DVNRM-HMF-WRE is $O(\left|{N}_{\mathrm{FN}}\right|+\left|{N}_{\mathrm{V}}\right|{\left|{N}_{\mathrm{S}}\right|}^{2}+k(\left|{N}_{\mathrm{V}}\right|+\left|{E}_{\mathrm{V}}\right|)(\left|{E}_{\mathrm{S}}\right|+\left|{N}_{\mathrm{S}}\right|\mathrm{lg}\left|{N}_{\mathrm{S}}\right|))$.

## 5. Simulation

In this section, we present the performance evaluation of the proposed DVNRM-HMF-WRE.

#### 5.1. Simulation Setup

We run our experiments on a workstation with Lenovo Tianyi 510Pro (Lenovo, Beijing, China) with Windows 10 system (Microsoft Corporation, Redmond, WA, USA). The hardware platform is the Inter Core i7-7700 3.6 GHz process (Intel Corporation, Santa Clara, CA, USA) with 8 GB RAM (Intel Corporation, Santa Clara, CA, USA). The software is MATLAB R2007a (MathWorks, Natick, MA, USA). We run our simulations for 3000 time units to leave the performance in a stable state. Each simulation is performed 10 times, and we take the average values as the final results. Simulation codes and results could be found in the Supplementary File.

The SN topology and VN topology used in simulation are generated by the improved Salam network topology random generation algorithm [24]. It is worth noting that, the main parameters of VN, SN, and failures are summarized in Table 2. In Table 2, [xmin, xmax] denotes a uniform distribution between xmin and xmax. Pois{p} and Expo{g} stand for the Poisson and exponential distributions, with mean p and g, respectively.

#### 5.2. Scenarios

Five scenarios are set in our simulation to validate the performance of DVNRM-HMF-WRE.

In the first scenario, the performance of DVNRM-HMF-WRE is compared with two typical VN reconfiguration methods TA-VNR [15] and ReNoVatE [18].

In the second scenario, our FVNRRM-WRE is compared with the other three faulty VN ranking methods to evaluate the impact of the FVNRRM-WRE. The details of them are shown in Table 3.

In the third scenario, our HMFRA is compared with the other two failure handling methods to evaluate the impact of the HMFRA. The details of them are shown in Table 4.

In the fourth scenario, our VNMM-WRE is compared with the other two VN node migration methods which include DVNRM-HMF-HOPS and DVNRM-HMF-RES. The details of them are shown in Table 5.

At last, the arrival rate of failure, u, and the lifetime of failure, T, are set to evaluate their effects on DVNRM-HMF-WRE.

#### 5.3. Simulation Results

#### 5.3.1. Comparison of Different VN Reconfiguration Methods

The results of comparison of faulty VN reconfiguration methods are shown in Figure 3.

As can be seen from Figure 3a–c, the DVNRM-HMF-WRE proposed in this paper has the best AR, NOR, and LAR/CR. It achieves 5.7% and 11.3% higher AR than TA-VNR and ReNoVatE, respectively. Its NOR is 6.9% and 11% higher than TA-VNR and ReNoVatE, respectively. Its LAR/CR is 25.7% and 41.9% higher than TA-VNR and ReNoVatE, respectively. In DVNRM-HMF-WRE, faulty components number, VN revenue, and remaining lifetime of faulty VN are all considered to rank the faulty VNs. The resource and topology indicators are both used based on WRE in VNMM-WRE. In addition, the faulty virtual node and VN link failures I are recovered first. Therefore, the performance of DVNRM-HMF-WRE is the best.

#### 5.3.2. Comparison of Faulty VN Ranking Methods

The results of comparison of faulty VN ranking methods are shown in Figure 4.

As can be seen from Figure 4a–c, the DVNRM-HMF-WRE achieves 4.5%, 5.3%, and 5.6% higher AR than DVNRM-HMF-FN, DVNRM-HMF-VNR, and DVNRM-HMF-RLT, respectively. The DVNRM-HMF-WRE achieves 3.1%, 6.4%, and 5.1% higher NOR than DVNRM-HMF-FN, DVNRM-HMF-VNR, and DVNRM-HMF-RLT, respectively. The DVNRM-HMF-WRE achieves 31.1%, 17.4%, and 25.1% higher LAR/CR than DVNRM-HMF-FN, DVNRM-HMF-VNR, and DVNRM-HMF-RLT, respectively. In this paper, our objective is to maximize the LAR/CR. If re-embedding the faulty VN fails, a high penalty cost will be paid. Improving the proportion of VN revenue, remaining lifetime of the faulty VN, and re-embedding the faulty VN which has higher penalty cost and revenue based on WRE, can improve the LAR/CR significantly.

#### 5.3.3. Comparison of Failure Handling Methods

The results of comparison of failure handling methods are shown in Figure 5.

As can be seen from Figure 5a–c, the performance of DVNRM-HMF-WRE is better than the other two failure handling methods. It achieves 5.7% and 7.3% higher AR than DVNRM-HMF-WRE II and DVNRM-HMF-WRE III, respectively. Its NOR is 8.5% and 12.8% higher than DVNRM-HMF-WRE II and DVNRM-HMF-WRE III, respectively. Its LAR/CR is 24% and 27.9% higher than DVNRM-HMF-WRE II and DVNRM-HMF-WRE III, respectively. When the substrate node is faulty, the virtual node and its connective virtual links (VN link failure I) will fail at the same time. Substrate node and link resources are both consumed to re-embed the faulty VN. Compared with the other two link failures, node failure is the hardest to recover. Therefore, re-embedding faulty virtual node and its connective virtual links, first, can improve the performance of VN recovery algorithm.

#### 5.3.4. Comparison of Different VN Node Migration Methods

The results of comparison of VN node migration methods are shown in Figure 6.

As can be seen from Figure 6a,b, DVNRM-HMF-WRE has the best AR and NOR. It achieves approximately 8.3% and 16.3% higher AR than VNMM-HMF-HOPS and VNMM-HMF-RES, respectively. The NOR of VNMM-HMF-WRE is approximately 7.4% and 17.4% higher than VNMM-HMF-HOPS and VNMM-HMF-RES, respectively. In Figure 6c, the advantage of VNMM-WRE is more obvious in LAR/CR. It is approximately 31.8% and 68.3% higher than VNMM-HMF-HOPS and VNMM-HMF-RES, respectively. In VNMM-HMF-WRE, the resource and topology indicators are both used to reduce substrate resource consumption in VN reconfiguration. Also, WRE is introduced to calculate the candidate substrate node importance.

#### 5.3.5. The Effect of Arrival Rate of Failure and Lifetime of Failure on DVNRM-HMF-WRE

In this section, the arrival rate of failure u and the lifetime of failure T are selected to describe their effects on DVNRM-HMF-WRE.

(1) The Arrival Rate of Failure u

In this section, the arrival rates of failure are set to 1/3, 1/6, and 1/9. The results of DVNRM-HMF-WRE in different arrival rates of failure are shown in Figure 7.

As can be seen from Figure 7a–c, with the increase of arrival rate of failure u, the number of SN failures keeps increasing, which results in a reduction in resources used for VN embedding and reconfiguration. Therefore, the AR and NOR of VN are decreasing. A large number of penalty cost caused by the failure of VN reconfiguration significantly reduces the LAR/CR.

(2) The Lifetime of Failure T

In this section, the lifetimes of failure are set to 400, 600, and 800. The results of DVNRM-HMF-WRE in different lifetimes of failure are shown in Figure 8.

As can be seen from Figure 8a–c, increasing the lifetime of failure can reduce the available substrate resources, which not only decreases the AR, but also results in a reduction in resources used for VN reconfiguration. Therefore, with the increase of the lifetime of failure, the AR, NOR, and LAR/CR are all reduced significantly.

## 6. Conclusions and Future Work

In this paper, we have addressed the problem of recovering faulty VNs affected by hybrid multiple failures in SN. In this regard, we have formulated the VN reconfiguration to an ILP model to maximize the LAR/CR. We have proposed an efficient heuristic policy called DVNRM-HMF-WRE to solve this ILP model. In DVNRM-HMF-WRE, the WRE method is introduced into FVNRRM-WRE and VNMM-WRE. In FVNRRM-WRE, the WRE method is used to rank the multiple faulty VNs. In VNMM-WRE, candidate substrate node are selected based on WRE method which takes resource and topology indicators into consideration. Also, a failure handling method called HMFRA is proposed, which handles virtual node failure and VN link failure I first. At last, five experiments are designed. The first experiment verifies that the proposed DVNRM-HMF-WRE has excellent performance than other typical VN reconfiguration methods. The next three experiments assess that our proposed faulty VN ranking method, failure handling method, and VN node migration method perform better than other corresponding methods. The last experiment sets two different scenarios to evaluate the performance of DVNRM-HMF-WRE in different arrival rates and lifetimes of failures. Evaluation results show that decreasing the arrival rate and the lifetime of failure can improve the performance of VN reconfiguration.

In future research, we will introduce the privacy and energy consumption into VN reconfiguration. Without adequate protection, users from a VN might be able to gain unauthorized access to data being transmitted through other VNs, violating the privacy of the entities that own those networks. Hence, privacy-oriented VNE and preserving privacy with VN stacks gradually get more and more attention. Also, energy consumption has become another hot topic in NV. With the increase of communication traffic every year, some energy models have been proposed to minimize the energy consumption in NV, such as the load-dependent power consumption model in VNE, the energy-efficient resource management for real-time service function chain in fog-supported SDN and network-aware energy optimization model for VN. Therefore, the ILP formulation and the heuristic algorithm could be extended to consider the privacy and energy consumption in future work.

## Supplementary Materials

supplementary materials are available online at https://www.mdpi.com/1099-4300/20/9/711/s1.

## Author Contributions

Y.S. performed initially the theoretical derivation, designed the experiments and software code, and wrote the draft of this paper in Chinese; X.M. proposed the original idea, checked the initial derivation, revised radically the paper, and rewrote the paper in English; Q.K. made a major contribution to the revision of this paper as for the reviews’ comments. X.H. analyzed the experimental data.

## Funding

This research was funded by [National Science Foundation of China] grant number [61401499].

## Conflicts of Interest

The authors declare no conflict of interest.

## Abbreviations

The following abbreviations are used in this manuscript:

NV | network virtualization |

VN | virtual network |

SN | substrate network |

SDN | software defined network |

ILP | integer linear programming |

DVNRM-HMF-WRE | dynamic virtual network reconfiguration method for hybrid multiple failures based on weighted relative entropy |

FVNRRM-WRE | faulty VN reconfiguration ranking method based on weighted relative entropy |

HMFRA | hybrid multiple failures ranking algorithm |

VNMM-WRE | virtual node migration method based on weighted relative entropy |

WRE | weighted relative entropy |

AR | acceptance ratio |

NOR | normal operation ratio |

LAR/CR | long-term average revenue to cost ratio |

## References

- Fischer, A.; Botero, J.F.; Beck, M.T.; de Meer, H.; Hesselbach, X. Virtual network embedding: A survey. IEEE Commun. Surv. Tutor.
**2013**, 15, 1888–1906. [Google Scholar] [CrossRef] - Zhang, J.; Zhao, C.; Wu, H.; Lin, M.; Duan, R. Virtual network embedding based on graph entropy. Entropy
**2016**, 20, 315. [Google Scholar] [CrossRef] - Mijumbi, R.; Serrat, J.; Gorricho, J.L.; Bouten, N.; Turck, F.D.; Boutaba, R. Network function virtualization: State-of-the-art and research challenges. IEEE Commun. Surv. Tutor.
**2016**, 18, 236–262. [Google Scholar] [CrossRef] - Tajiki, M.M.; Shojafar, M.; Akbari, B.; Salsano, S.; Conti, M.; Singhal, M. Joint failure recovery, fault prevention, and energy-efficient resource management for real-time SFC in fog-supported SDN. Comput. Netw. Available online: http://cn.arxiv.org/pdf/1807.00324 (accessed on 1 July 2018).
- Shojafar, M.; Canali, C.; Lancellotti, R.; Baccarelli, E. Minimizing computing-plus-communication energy consumptions in virtualized networked data centers. In Proceedings of the IEEE Symposium on Computers and Communication (ISCC), Messina, Italy, 27–30 June 2016; pp. 94–104. [Google Scholar]
- Shahriar, N.; Chowdhury, S.R.; Ahmed, R. Virtual network survivability through joint space capacity allocation and embedding. IEEE J. Sel. Areas Commun.
**2018**, 36, 502–518. [Google Scholar] [CrossRef] - Boem, F.; Ferrari, R.M.G.; Keliris, C.; Parisini, T.; Polycarpou, M. A distributed networked approach for fault detection of large-scale systems. IEEE Trans. Autom. Control
**2016**, 62, 18–33. [Google Scholar] [CrossRef][Green Version] - Herker, S.; Khan, A.; An, X. Survey on survivable virtual network embedding problem and solutions. In Proceedings of the 9th International Conference on Networking and Services (ICNS), Lisbon, Portugal, 24–29 March 2013; pp. 94–104. [Google Scholar]
- Li, R.; Wu, Q.; Tan, Y.; Zhang, J. On the optimal approach of survivable virtual network embedding in virtualized SDN. IEICE Trans. Inf. Syst.
**2018**, 101, 698–708. [Google Scholar] [CrossRef] - Sangjin, H.; Jason, P.J.; Pyungkoo, P.; Hosun, Y.; Hoyong, R.; Sungback, R. Survivable virtual topology design in multi-domain optical networks. J. Opt. Commun. Netw.
**2016**, 8, 408–416. [Google Scholar] [CrossRef] - Aguado, A.; Davis, M.; Peng, S.; Alvarez, M.V.; Lopez, V.; Szyrkowiec, T.; Autenrieth, A.; Vilalta, R.; Mayoral, A.; Muñoz, R.; et al. Dynamic virtual network reconfiguration over SDN orchestrated multi-technology optical transport domains. J. Lightw. Technol.
**2016**, 34, 1933–1938. [Google Scholar] [CrossRef][Green Version] - Chowdhury, S.R.; Ahmed, R.; Khan, M.M.A.; Shahriar, N.; Boutaba, R.; Mitra, J.; Zeng, F. Dedicated protection for survivable virtual network embedding. IEEE Trans. Netw. Serv. Manag.
**2016**, 13, 913–926. [Google Scholar] [CrossRef] - Ayoubi, S.; Chen, Y.; Assi, C. Towards promoting backup-sharing in survivable virtual network design. IEEE/ACM Trans. Netw.
**2016**, 24, 3218–3231. [Google Scholar] [CrossRef] - Chang, X.; Muppala, J.K.; Wang, B.; Liu, J.; Sun, L. Migration cost aware virtual network re-embedding in presence of resource failures. In Proceedings of the 18th IEEE International Conference on Networks (ICON), Singapore, 12–14 December 2012; pp. 24–29. [Google Scholar]
- Peng, L. A topology-awareness virtual network reconfiguration algorithm. J. Sichuan Univ.
**2015**, 47, 110–115. [Google Scholar] [CrossRef] - Ghaleb, A.M.; Khalifa, T.; Ayoubi, S.; Shaban, K.B. Surviving link failures in multicast VN embedded applications. In Proceedings of the IEEE/IFIP Network Operations and Management Symposium, Istanbul, Turkey, 25–29 April 2016; pp. 645–651. [Google Scholar]
- Qiang, Z.; Qiang, W.; Sheng, F.; Wu, L. Heuristic survivable virtual network embedding based on node migration and link remapping. In Proceedings of the IEEE 7th Joint International Information Technology and Artificial Intelligence Conference, Chongqing, China, 20–21 December 2014; pp. 181–185. [Google Scholar]
- Shahriar, N.; Ahmed, R.; Khan, A.; Chowdhury, S.R.; Boutaba, R.; Mitra, J.; David, R. ReNoVatE: Recovery from node failure in virtual network embedding. In Proceedings of the 12th International Conference on Network and Service Management (CNSM), Montreal, QC, Canada, 31 October–4 November 2016; pp. 1–9. [Google Scholar]
- Shahriar, N.; Ahmed, R.; Chowdhury, S.R.; Khan, A.; Boutaba, R.; Mitra, J. Generalized recovery from node failure in virtual network embedding. IEEE Trans. Netw. Serv. Manag.
**2017**, 14, 261–274. [Google Scholar] [CrossRef] - Gong, S.; Chen, J.; Zhao, S.; Zhu, Q. Virtual network embedding with multi-attribute node ranking based on TOPSIS. KSII Trans. Int. Inf. Syst.
**2016**, 10, 522–541. [Google Scholar] [CrossRef] - Su, Y.; Meng, X.; Meng, Q.; Zhao, Z. Environment adaptive and joint topology aware virtual network embedding algorithm. J. Electr. Inf. Technol.
**2018**, 40, 79–86. [Google Scholar] [CrossRef] - Zhu, Y.; Ammar, M. Algorithms for assigning substrate network resources to virtual network components. In Proceedings of the 25th IEEE International Conference on Computer Communications, Barcelona, Spain, 23–29 April 2006; pp. 1–12. [Google Scholar]
- Gong, S.; Chen, J.; Huang, C.; Zhu, Q. Trust-aware secure virtual network embedding algorithm. J. Commun.
**2015**, 36, 180–189. [Google Scholar] [CrossRef] - Zhao, Z.; Meng, X.; Su, Y.; Li, Z. Virtual network embedding based on node connectivity awareness and path integration evaluation. KSII Trans. Int. Inf. Syst.
**2017**, 11, 3393–3412. [Google Scholar] [CrossRef]

**Figure 1.**Examples of VN embedding and reconfiguration. (

**a**) The VNs; (

**b**) The process of VNE; (

**c**) The process of VN reconfiguration for VN node failure, VN link failure I and II; (

**d**) The process of VN reconfiguration for VN link failure III.

**Figure 3.**Comparison of DVNRM-HMF-WRE, TA-VNR and ReNoVatE. (

**a**) Acceptance ratio; (

**b**) Normal operation ratio; (

**c**) Long-term average revenue to cost ratio.

**Figure 4.**Comparison of different faulty VN ranking methods. (

**a**) Acceptance ratio; (

**b**) Normal operation ratio; (

**c**) Long-term average revenue to cost ratio.

**Figure 5.**Comparison of different failure handling methods. (

**a**) Acceptance ratio; (

**b**) Normal operation ratio; (

**c**) Long-term average revenue to cost ratio.

**Figure 6.**Comparison of different VN node migration methods. (

**a**) Acceptance ratio; (

**b**) Normal operation ratio; (

**c**) Long-term average revenue to cost ratio.

**Figure 7.**Comparison of DVNRM-HMF-WRE in different arrival rates of failure. (

**a**) Acceptance ratio; (

**b**) Normal operation ratio; (

**c**) Long-term average revenue to cost ratio.

**Figure 8.**Comparison of DVNRM-HMF-WRE performance in different arrival rates of failure. (

**a**) Acceptance ratio; (

**b**) Normal operation ratio; (

**c**) Long-term average revenue to cost ratio.

Notation | Description | Notation | Description |
---|---|---|---|

G_{S} | SN | G_{V} | VN |

N_{S} | Substrate node set | N_{V} | Virtual node set |

E_{S} | Substrate link set | E_{V} | Virtual link set |

|N_{S}| | Number of substrate nodes | |N_{V}| | Number of virtual nodes |

|E_{S}| | Number of substrate links | |E_{V}| | Number of virtual links |

n_{s} | Substrate node | n_{v} | Virtual node |

e_{s} | Substrate link | e_{v} | Virtual link |

cpu(n_{s}) | CPU resource of n_{s} | cpu(n_{v}) | Required CPU resource of n_{v} |

bw(e_{s}) | Bandwidth resource of e_{s} | bw(e_{v}) | Required bandwidth resource of e_{v} |

loc(n_{s}) | Location attribute of n_{s} | N_{Vf} | Faulty virtual node set |

N_{S}_{f} | Faulty substrate node set | E_{Vf} | Faulty virtual link set |

E_{S}_{f} | Faulty substrate link set | T_{V} | Lifetime of VN |

G_{Vf} | Faulty VN set | F_{i,j} | Failure events |

F_{1,j} | Substrate node failure | F_{2,j} | Substrate link failure |

t_{e}(F_{i,j}) | Ending time of F_{i,j} | t_{s}(F_{i,j}) | Starting time of F_{i,j} |

VN_{map}(t) | VNs which are embedded successfully at time t | |VN_{map}(t)| | Number of VNs which are embedded successfully at time t |

R(G_{V}, t) | Revenue of G_{V} at time t | VN_{fr}(t) | Set of faulty VNs which are re-embedded unsuccessfully |

C(G_{V}, t) | Cost of G_{V} at time t | PCost(G_{V}, t) | Penalty cost of G_{V} at time t |

hops(e_{v}) | Hop counts in substrate link corresponding to virtual link e_{v} | |VN_{fr}(t)| | Number of faulty VNs which are re-embedded unsuccessfully |

|VN(t)| | Total number of VN requests at time t | η(t) | Normal operation ratio at time t |

x(n_{v},n_{s})∈{0,1} | x(n_{v}, n_{s}) = 1 if virtual node n_{v} is embedded onto substrate node n_{s} | x(e_{up}, e_{ij})∈{0,1} | x(e_{up}, e_{ij}) = 1 if virtual link e_{up} is embedded onto substrate link e_{ij} |

s(n_{v})∈{0,1} | s(n_{v}) = 1 if n_{v} is faulty | s(n_{s})∈{0,1} | s(n_{s}) = 1 if n_{s} is faulty |

s(e_{v})∈{0,1} | s(e_{v}) = 1 if e_{v} is faulty | s(e_{s})∈{0,1} | s(e_{s}) = 1 if e_{s} is faulty |

s(G_{V}) ∈{0,1} | s(G_{V}) = 1 if G_{V} is faulty | s(G_{S})∈{0,1} | s(G_{S}) = 1 if G_{S} is faulty |

dis(loc(n_{s}), loc(n_{v})) | Distance between n_{s} and n_{v} | A_{i} | State of system A |

B_{i} | State of system B | C | Entropy value |

X_{ij} | The j-th indicator coefficient of the ith node | X | Decision matrix |

ω_{j} | Weighting coefficient of the j-th indicator | A^{+} | Positive ideal solutions |

A^{-} | Negative ideal solutions | Z_{i} | Similarity between each solution and the ideal one |

Z(n_{v}) | Importance degree of virtual node n_{v} | Z(n_{s}) | Importance degree of substrate node n_{s} |

Can(n_{vi}) | Candidate substrate node set | |N_{FN}| | Total number of faulty VNs |

Characteristics | Values | Characteristics | Values |
---|---|---|---|

Number of substrate node | 100 | Number of virtual node per VN | [5, 10] |

Substrate node capacity | [50, 100] | Virtual node capacity | [2, 10] |

Number of substrate link | 500 | Virtual link connection probability | 0.5 |

Substrate link capacity | [50, 100] | Virtual link capacity | [2, 10] |

VN arrive rate | Pois{0.1} | VN lifetime | Expo{300} |

Failure arrive rate | Pois{0.2} | Failure lifetime | Expo{500} |

Network distance scope | 1000 × 1000 | Virtual node position constrain | 400 |

Algorithm | Description |
---|---|

DVNRM-HMF-WRE | VN reconfiguration method proposed in this paper which ranks faulty VNs based on FVNRRM-WRE. |

DVNRM-HMF-FN | VN reconfiguration method which ranks faulty VNs based on faulty component number. |

DVNRM-HMF-VNR | VN reconfiguration method which ranks faulty VNs based on VN revenue. |

DVNRM-HMF-RLT | VN reconfiguration method which ranks faulty VNs based on remaining lifetime of the faulty VN. |

Algorithm | Description |
---|---|

DVNRM-HMF-WRE | VN reconfiguration method proposed in this paper which handles different failures based on HMFRA. |

DVNRM-HMF-WRE II | VN reconfiguration method which handles VN link failure II first. |

DVNRM-HMF-WRE III | VN reconfiguration method which handles VN link failure III first. |

Algorithm | Description |
---|---|

DVNRM-HMF-WRE | VN reconfiguration method proposed in this paper, in which virtual node is migrated based on VNMM-WRE. |

DVNRM-HMF-RES | VN reconfiguration method in which virtual node is migrated based on the total node and link resources. |

DVNRM-HMF-HOPS | VN reconfiguration method in which virtual node is migrated based on the reciprocal of hop counts between candidate substrate node and substrate node which is embedded by the neighboring virtual node of the migrated virtual node. |

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).