An Efficient Local Search for the Feedback Vertex Set Problem

Inspired by many deadlock detection applications, the feedback vertex set is defined as a set of vertices in an undirected graph, whose removal would result in a graph without cycle. The Feedback Vertex Set Problem, known to be NP-complete, is to search for a feedback vertex set with the minimal cardinality to benefit the deadlock recovery. To address the issue, this paper presents NewkLS FVS(LS, local search; FVS, feedback vertex set), a variable depth-based local search algorithm with a randomized scheme to optimize the efficiency and performance. Experimental simulations are conducted to compare the algorithm with recent metaheuristics, and the computational results show that the proposed algorithm can outperform the other state-of-art algorithms and generate satisfactory solutions for most DIMACSbenchmarks.


Introduction
Inspired by many deadlock detection applications, the Feedback Vertex Set Problem (FVSP) is known to be NP-complete and plays an important role in the study of deadlock recovery [1,2].For example, the wait-for graph is a directed graph used for deadlock detection in operating systems and relational database systems of an operating system.and each directed cycle corresponds to a deadlock situation in the wait-for graph.In order to resolve all deadlocks, some blocked processes need to be aborted.
A minimum feedback vertex set in this graph corresponds to a minimum number of processes that one needs to abort.Therefore solving the FVSP with more efficiency and better performance can contribute to an improved deadlock recovery.
To describe the FVSP, the following concepts are predefined.Assuming G = (V (G), E(G)) is a graph, then the set of vertices is denoted by V (G), and the set of edges of G is denoted by E(G).For S ⊆ V (G), the subgraph induced by S is denoted by G [S].The set of vertices adjacent to a vertex, i ∈ V (G), will be denoted by N (i) = {j ∈ V (G) : (i, j) ∈ E} and called the openneighborhood of the vertex, i.Let G be a graph.A feedback vertex set, S, in G is a set of vertices in G, whose removal results in a graph without cycle (or equivalently, every cycle in G contains at least one vertex in S).
The FVSP can be considered in both directed and undirected graphs.Let G be an undirected graph.A Feedback Vertex Set Problem (FVSP), S, in G is a set of vertices in G, whose removal results in a graph without cycle (or equivalently, every cycle in G contains at least one vertex in S).Let G be a directed graph.A feedback vertex set (FVS), S, in G is a set of vertices in G, whose removal results in a graph without directed cycle (or equivalently, every directed cycle in G contains at least one vertex in S).
Inspired by real world applications, several variants were proposed over the years.Some of them are summarized by the following definitions.
(a) FVSP: Given a graph, G, the feedback vertex set problem is to find an FVS with the minimum cardinality.
(b) The Parameterized Feedback Vertex Set Problem (DFVSP): Given a graph, G, and a parameter, k, the Parameterized Feedback Vertex Set Problem is to either find an FVS of at most k vertices for G or report that no such set exists.
(c) The Vertex Weighted Feedback Vertex Set Problem (VWFVSP): Given a vertex weighted graph, G, the Vertex Weighted Feedback Vertex Set Problem is to find an FVS with the minimum weight.
The Parameterized Feedback Vertex Set Problem is the decision version of the feedback vertex set problem.The FVSP and DFVSP were classic NP-complete problems that appeared in the first list of NP-complete problems in Karp ′ s seminal paper [3].When the edges or arcs instead of vertices are considered, it becomes the corresponding edge or arc version.For example, the parameterized feedback arc set problem is to find out whether there is a minimum of k arcs in a given graph, whose removal makes the graph acyclic.
Let G be a graph.It is said that a vertex set, S, is an Without loss of generality, only the feedback vertex set problem for an undirected graph is considered in this paper, and the aim is to search for the largest subset, S ⊆ V (G), such that G[S] is acyclic.
The rest of this paper is organized as follows: Section 2 presents the related work of local search heuristics.Then, Sections 3 and 4 propose the k-opt local search (KLS)-based local search algorithm and the KLS with a randomized scheme algorithm in detail.Section 5 conducts the experiments of both the proposed algorithms and the compared metaheuristics with the results for the performance and efficiency evaluation.Finally, Section 6 discusses the contributions of this paper and draws a conclusion.

Related Work
As a local improved technology, the local search is a practical tool and a common technique looking for the near-optimal solutions for combinatorial optimization problems [4].In order to obtain highquality solutions, it was applied to many metaheuristic algorithms, such as simulating annealing, genetic algorithm, memetic algorithm and swarm intelligent algorithms, in many cases.
The local search is a common tool looking for near-optimal solutions in reasonable time for combinatorial optimization problems.Usually, the current solution is iteratively replaced by an improved solution from its neighborhood, until no better solution can be obtained.Then, the current solution is called locally optimal.
The basic local search starts with a feasible solution, x, and repeatedly replaces x with an improved one, x ′ , which is selected from the neighborhood of x.If no better neighbor solutions can be found in its neighborhood, the local search immediately stops and returns as the final best solution found during the search [4].
In many cases, the local search can be applied into heuristic algorithms, such as simulated annealing, ant colony and particle swarm optimizations.However, normally, it is hard to obtain the best solution, or even an approximate solution with high quality, for a certain combinatorial optimization problem.Therefore, various local search methods, such as phased local search [19] and variable depth local search, were proposed.The variable depth local search was initially used to solve the Graph Partitioning Problem (GPP) [9] and the Traveling Salesman Problem (TSP) [8].Then, it was applied to other heuristic algorithms [20][21][22][23].In [4], it was applied to solve the maximum clique problem and successfully obtained good solutions.
Different from the previous studies, the proposed local search algorithms are based on KLS and are combined with a depth variable search, which can improve the performance and efficiency.

KLS-Based Local Search Algorithm
In the simple local search procedure, the current solution is changed by adding or removing one vertex.The idea of KLS-based local search is to operate more vertices for each time.That is to say, the current solution is changed by way of adding or removing more than one vertex.Furthermore, the number of operated vertices is not limited in KLS.Therefore, it is a variable depth local search, also called k-opt local search.
As a comparison, the KLS-based local search is explored to solve the feedback vertex set problem.Then, a variable depth-based local search with a randomized scheme is proposed to solve this problem.For a given graph, G, it can be observed that if S is acyclic, then V (G) \ S is a feedback vertex set.Therefore, such a vertex set, S, with maximum cardinality corresponds to a feedback vertex set with minimum cardinality.Hence, the goal is to find a vertex set, S ∈ V (G), with maximum cardinality, for which G[S] is acyclic.The main framework of this algorithm is taken from KLS in [4].
The following notations will be used to describe the algorithms.
• CF : the current vertex subset, CF , for which G[CF ] is acyclic.• P A: the possible vertex set of addition.i.e., P A = {v|the subgraph induced by CF ∪ {v} is acyclic} First, a vertex set, CF ⊆ V , is randomly generated as an initial acyclic vertex set.If CF is not maximal, then a vertex, v ∈ P A, that minimizes f (v) is chosen to add to CF .On the contrary, if CF is maximal, then a vertex, v ∈ CF , with the maximum degree is chosen to be dropped.In the search process, some vertices are iteratively added or removed, ensuring that the vertex set, CF , is acyclic at each iteration.In order to avoid cycling, when a vertex, v, is added (resp.dropped),v would not be dropped (resp.added) immediately.The pseudocode of the procedure is shown in Algorithm 1 (k-opt-LS).until D = ∅ 18: until g max ≤ 0 In this local search algorithm, if P A = ∅, then the acyclic vertex set, CF , is maximal.In this case, a vertex, v ∈ CF , will be dropped; otherwise, a vertex v ∈ V \ CF will be added.Algorithm 1 starts with a random acyclic vertex set, CF ; then, a possibly better acyclic vertex set is obtained by adding or removing more than one vertex.Therefore, it is called the variable depth local search, and it can be applied to other heuristic algorithms, such as simulated annealing.Compared with the simple local search algorithm, where only one vertex would be added or removed to change the current solution, the variable depth local search operates more vertices and provides more efficiency in searching for the desired acyclic vertex set.

KLS with a Randomized Scheme for the FVSP
In Algorithm 1, when a vertex is added (resp.dropped), it is no longer dropped (resp.added) if it does not exit the inner loop.Moreover, at each iteration, a vertex that minimizes f + (v) or minimizes f (v) is selected.These conditions are too restricted, and they may trap the procedure into a local minimum.In order to overcome these drawbacks, a variable T v is used to store the moment of the last iteration, while the vertex, v, was under operation, and c is used to record the iteration number of the inner loop.If v is added (resp.dropped), then the chance is given to make v dropped (resp.added) for at most k times within a given number of iterations.Moreover, at each iteration, a vertex is selected with a probability associated with the degree of v.The modified version of Algorithm 1 is shown in Algorithm 2 (NewKLS FVS(LS, local search; FVS, feedback vertex set)).

Algorithm 2 NewKLS FVS(G, CF, P A, k)
Require: G: a graph with the vertex set, {1, 2, • • • , n}; CF : the current acyclic vertex set; P A: the possible vertex set of addition; 1: generate an acyclic vertex set, CF , randomly.2: repeat 3: find a vertex, v, from P A ∩ P with a probability, ρ. 9: find a vertex v from CF ∩ P with a probability, ρ. 13: if v is contained in CF P rev then until D = ∅ 20: until g max ≤ 0

Experimental Setup and Benchmark Instances
All the algorithms are coded in Visual C++ 6.0 and executed in an Intel Pentium(R) G630 Processor 2.70 MHz with 4 GB of RAM memory on a Windows 7 Operating System.
Two sets of testing instances are used in the experiments.One instance is a set of random graphs, which are constructed with parameter n and p with order (the vertex number) n, inserting an edge with probability p.This probability is called the edge density (or edge probability).The graph, G(n, p), is used to represent a graph constructed with parameter n and p.Then, the proposed algorithm is compared with other algorithms on random graphs, G(400, 0.5) and G(800, 0.5).The other instance is the wellknown DIMACSbenchmarks for graph coloring problems (see [24]), consisting of 59 graph coloring problem instances.Table 1 gives the characteristics of these instances, including the number of vertices (|V |), the number of edges (|E|) and the graph density (density).In order to observe the impact of the tabu tenure of the algorithm, NewKLS FVS, the algorithm for the tabu tenure, k, is tested from 100 to 1,000 on four chosen graphs.From the results above, it can be seen that the parameter, k, is important for the algorithm.For example, for the graph, inithx.i.1.col(resp.le45025a.col,G(400,0.5),G(800,0.5)),k = 900 is the most suitable one.

Comparison of NewKLS FVS with Other Algorithms
In order to verify the efficiency and performance of the proposed algorithm, the performance of the proposed heuristic for the feedback vertex set problem is also compared to that of several others.Note that the proposed problem can be solved approximately by various heuristics, including simulated annealing (SA) and variable neighborhood search (VNS).
(1) Simulated annealing: The simulated annealing algorithm was introduced to solve combinatorial optimization problems proposed by Kirkpatrick [25].Since then, it has been widely investigated to solve many combinatorial optimization problems.Simulated annealing allows a transition to go opposite towards achieving the goal with a probability.As a result, the simulated annealing algorithm has some opportunities to jump out of the local minima.As preparation, for a given graph, G, and a positive integer, k, a partition Then, the parameters, T 0 (initial temperature), T s (stop temperature) and α (cooling down coefficient), are required as input.For convenience, the edge numbers of G[S 1 ] and G[S ′ 1 ] are denoted by e(S 1 ) and e(S ′ 1 ), respectively.After the preparation, the algorithm begins with a randomly generated partition (S 1 , S 2 ) of V (G).Then, a partition (S ′ 1 , S ′ 2 ) is chosen that is obtained by swapping two elements from S 1 and S 2 .If e(S ′ 1 ) is fewer than e(S 1 ), then (S ′ 1 , S ′ 2 ) is accepted.Otherwise, it is accepted according to the simulated annealing rule.At the end of each iteration, the temperature would cool down.Usually, let T = αT , where α ∈ (0, 1).If G[S 1 ] is acyclic, the procedure terminates successfully, and an acyclic induced subgraph with order k would be returned.When the temperature reaches T s , the procedure also terminates.The pseudocode is presented in Algorithm 3, where u(0, 1) denotes a random number in (0, 1).Algorithm 3 k, T 0 , T s , α,t) T ← αT ; 14: end while 15: end while (2) Variable neighborhood search: Hansen and Mladenović [10][11][12] introduced the variable neighborhood search (VNS) method in combinatorial problems.In [5], it was applied to solve the maximum clique problem.It constructs different neighborhood structures, which are used to perform a systematic search.The main idea of variable neighborhood search is that when it does not find a better solution for a fixed number of iterations, the algorithm continues to search in another neighborhood until a better solution is found.
as a finite set of pre-selected neighborhood structures and N k (X) as the set of solutions in the kth neighborhood of X.The steps of the basic VNS are presented in Algorithm 4 (see [5]).
To solve the feedback vertex set problem, the neighborhood structure is described as follows.Let (S 1 , S 2 ) be the current partition, for a positive integer, k, as the set of the k-subset of S. The kth neighborhood of P is defined by: The proposed algorithms operate the partition, S 1 , S 2 .They change the current partition to another solution, S ′ 1 , S ′ 2 , by swapping exactly k vertices between S 1 and S 2 .Therefore, the neighborhood of (S 1 , S 2 ) is the set of all the solutions obtained from (S 1 , S 2 ) by swapping exactly k vertices between S 1 and S 2 .The parameter, k max , called the radius of the neighborhood, controls the maximum k of the procedure.
The algorithms, k-opt-LS, NewKLS FVS, VNS and SA, are carried out on several random graphs and DIMACS benchmarks for the maximum feedback vertex set.In the VNS algorithm, the value of K max is set to be one and two, which are called VNS1 (if K max = 1) and VNS2 (if K max = 2), respectively.In the SA algorithm, the initial temperature is set to be T 0 = 1, 000, and the cooling down coefficient α = 0.9997.Tables 3-7 show the results of these algorithms, and each is carried out for 15 runs with a CPU-time limit of 30 min.

Algorithm 4 VNS(G, r)
initial solution X and stop criteria; 2: while Stop condition is not met do Apply the local search method with X ′ as the initial solution; denote by X ′′ the obtained solution; In order to show the performance and efficiency of various algorithms, experiments are conducted to test the algorithms (NewkLS FVS, k-opt-LS, VNS1, VNS2, SA) on graphs huck.col,inithx.i.1.col,le450 25a.col, zeroin.i.1.coland school1 nsh.col.The convergence curves of these algorithms are reported on both running times (with a CPU-time limit of 10 min; see Figure 1) and iteration numbers (within 100 iterations; see Figure 2).The experimental results show that NewkLS FVS outperforms other algorithms on graphs huck.col,le450 25a.col, zeroin.i.1.coland school1 nsh.col almost at any moment.The solution quality is better than any other compared algorithm after 3 min on inithx.i.1.col.Additionally, the solution quality of NewkLS FVS is better than those of any other compared algorithm on all tested graphs after 20 iterations.It can be seen that the performance of the algorithm, k-opt-LS, is better than VNS1, VNS2 and SA on almost all these instances, and the convergence performance of k-opt-LS is improved by using the proposed approaches.

Evaluation of the Algorithms
From the report of Tables 3-7, it can be seen that the results of NewKLS FVS and k-opt-LS are obviously better than those of VNS1, VNS2 and SA.Table 8 presents the results of the best algorithm for each instance testing DIMACS.In Table 8, the column best means the best result from the algorithms, and the column worst is the worst result from the algorithms.The column the-best-algorithm is the set of algorithms capable of obtaining the best result.Table 8 shows that NewKLS FVS obtains the best result for all the tested instances, and k-opt-LS obtains 34 best results.From the report, it can be seen that they both perform better than other test procedures for the feedback vertex set problem.

Concluding Remarks
This paper addresses an NP-complete problem, the feedback vertex set problem, which is inspired by many applications, such as deadlock detection in operating systems and relational database systems of an operating system.An efficient local search algorithm, NewkLS FVS, is proposed to solve this problem, and it was compared with popular heuristics, such as variable depth search and simulated annealing.From the experiments on random graphs and DIMACS benchmarks, it can be seen that NewkLS FVS is able to obtain better solutions than the variable depth search, and it has good performance by setting the parameter, tabu tenure, to a suitable value.

Table 1 .
Benchmark instances and their characteristics.

Table 2 .
Table 2 reports the best and worst solutions, respectively, for the algorithm, NewKLS FVS, with different values of the parameter, k. Results with with different values of the parameter k.

1 :
while Stop condition is not met do Generate a partition (S 1 , S 2 ) of V (G) with |S 1 | = k;

8 :
if X ′′ is better than incumbent then

Table 3 .
Results for NewKLS FVS (LS, local search; FVS, feedback vertex set) and a CPU-time limit of 30 min for 15 runs.

Table 4 .
Results for k-opt-LS and a CPU-time limit of 30 min for 15 runs.

Table 5 .
Results for variable neighborhood search 1 (VNS1) with k max = 1 and a CPU-time limit of 30 min for 15 runs.

Table 6 .
Results for VNS2 with k max = 2 and a CPU-time limit of 30 min for 15 runs.

Table 7 .
Results for simulated annealing (SA) and a CPU-time limit of 30 min for 15 runs.

Table 8 .
Results for algorithm analysis.