The Use of an Exact Algorithm within a Tabu Search Maximum Clique Algorithm

: Let G = ( V , E ) be an undirected graph with vertex set V and edge set E . A clique C of G is a subset of the vertices of V with every pair of vertices of C adjacent. A maximum clique is a clique with the maximum number of vertices. A tabu search algorithm for the maximum clique problem that uses an exact algorithm on subproblems is presented. The exact algorithm uses a graph coloring upper bound for pruning, and the best such algorithm to use in this context is considered. The ﬁnal tabu search algorithm successfully ﬁnds the optimal or best known solution for all standard benchmarks considered. It is compared with a state-of-the-art algorithm that does not use exact search. It is slower to ﬁnd the known optimal solution for most instances but is faster for ﬁve instances and ﬁnds a larger clique for two instances.


Introduction
Exact algorithms for the maximum clique problem are now remarkably efficient, but, for larger problems, heuristic algorithms are necessary. Tabu search is an effective metaheuristic for the maximum clique problem. This paper evaluates the option of including an exact algorithm within the tabu search, and considers the best exact algorithm to use. The candidate exact algorithms all use a graph coloring upper bound. This approach was used in the specific context of permutation code constructions in [1]. The final tabu search algorithm described, modified from a published algorithm [2], finds best known solutions to standard instances but is normally slower than a state-of-the-art tabu search algorithm [3] to find these solutions. However, it is more competitive for harder instances and actually finds significantly larger cliques for two open instances derived from the construction of permutation codes.
Let G = (V, E) be an undirected graph with vertex set V and edge set E. A clique C of G is a subset of the vertices of V with every pair of vertices of C adjacent (a complete subgraph). A maximum clique is a clique with the maximum number of vertices. Problems involving cliques arise in many applications surveyed in [4], including bioinformatics, examination planning, location problems, signal transmission analysis, and social network analysis. Applications of the maximum clique problem in radio frequency assignment are described in [5]. The problem also arises in the construction of various types of error-correcting code with the maximum number of codewords [6,7]. Note that the problem of finding a maximum clique in the graph G is equivalent to the problem of finding a maximum independent set in the complement graph G.
The decision form of the clique problem (does there exist a clique of size k) was proved to be one of the original 21 NP-complete problems in a classic paper [8]. It follows that the maximum clique problem is NP-hard. When exact methods are impractical, heuristic methods are used.
Algorithms for the maximum clique problem, both exact and heuristic, are surveyed in [4]. The algorithm Swap-Based Tabu Search (SBTS) presented in [3] is the only heuristic algorithm compared in [4] that is able to find the optimal or best known solution over a wide range of instances. A more recent algorithm HTS (Hybrid Tabu Search) [2] also finds the optimal or best known solution over these instances and adds some further instances based on the construction of permutation codes [7]. HTS makes use of both an exact and a pseudoexact inner solver. In the tabu search in HTS, a number of carefully selected vertices are removed from the current clique before an exact or pseudoexact algorithm is applied to the subgraph induced by the set of vertices of G adjacent to all vertices in the reduced clique.
As well as tabu search and other local search methods, there are also mathematical programming approaches, together with other methods, such as genetic algorithms, ant colony optimization, chemical reaction optimization, and particle swarm optimization. Hybrid and parallel versions of these algorithms also exist. The reader is referred to the survey [4] for an extensive list of references. Some later references to the maximum clique and related problems can be found in [9][10][11][12][13]. When comparable, these later approaches do not always match SBTS or HTS. For this reason, the current paper concentrates on tabu search. Although there are many effective algorithms, there is always scope for improvement on large and hard instances beyond the standard benchmarks.
There are four objectives of this paper. The first is to determine the best exact algorithm to replace the exact and pseudoexact algorithms used in HTS. This allows a comparison of the resulting tabu search algorithm, incorporating the best exact algorithm found, with a more standard tabu search approach. Initially, a comparison of various exact algorithms applied to benchmarks used in the comparison in [4] is carried out. The best candidates are then incorporated into the HTS algorithm and further compared, leading to a somewhat different rank ordering of methods. The second objective is to consider and implement a small number of other improvements to HTS. The third is to dramatically simplify the number of parameters in HTS by finding methods to determine most automatically. Finally, the resulting new algorithm HTS2 (Code for HTS2 is available in the Supplementary Material) can be compared with the original HTS and SBTS on the same machine. The overall aim of the work is to determine whether a tabu search algorithm including exact search can match SBTS on standard instances and find larger cliques than SBTS for some hard instances. This requires consideration of some new hard instances beyond those used in [3]. The new instances here are motivated by the authors' interest in the construction of permution codes.

Exact Algorithms with a Coloring Bound
Exact algorithms for the maximum clique problem are surveyed in [4]. The basis for many improved algorithms is the algorithm of Carraghan and Pardalos [14], and this is also the basis for the exact and pseudoexact algorithms used in HTS [2]. Within HTS,this algorithm works with a current clique F and a potential expansion set S = N(F) of vertices adjacent to all vertices of F. Vertices of S are selected in turn and removed from S. The new potential expansion set S is computed and the selected vertex is added to F to create a set F . If the potential expansion set is empty, and a new best clique is obtained, the clique is assigned to a global variable Best. Otherwise, subject to a pruning condition, the algorithm is applied recursively to the new potential expansion set S . The usual condition is that pruning of the search tree takes place unless |F | + |S | is greater than the size of the best clique found so far in the exact algorithm. Within HTS, extra pruning is used involving an external lower bound arising from the best clique found so far in the metaheuristic. Pseudoexact search uses a probable upper bound for pruning derived from an evaluation of typical subproblems solved for the instance within HTS.
A major improvement arises from the use of a vertex coloring bound.

Definition 1.
A vertex coloring of a graph G is an assignment of colors to the vertices of G such that adjacent vertices are assigned different colors.

Definition 2.
The chromatic number of a graph G is the minimum number of colors in a vertex coloring.
The following proposition is well known: The chromatic number of a graph G is an upper bound for the number of vertices in a maximum clique.
Proof. Every vertex of the clique must be assigned a different color.
The colors can be represented as positive integers 1, 2, . . .. Thus, any upper bound k for the chromatic number of the subgraph induced by S (obtained from a coloring of S with k colors) can replace |S | in the pruning to reduce the size of the search, sometimes dramatically. Then, options to be considered here are i) the best coloring algorithm to use, which must be very fast as the exact algorithm is used many times in hybrid tabu search, ii) whether to locally color each subgraph induced by S , which may be expensive, or to color the initial subproblem and use the colors inherited by the subgraph induced by S , or a combination of the two, iii) whether to re-order the vertices of S in non-increasing order of color within the exact solver.

Initial Evaluation of Exact Algorithms with a Coloring Bound
Improvements to the Carraghan and Pardalos algorithm (denoted as "C&P") using a coloring bound are of three basic types. The first, denoted as "local coloring", applies a coloring algorithm to the original subproblem and to each new set S created as the depth of the search tree is increased. Clearly the coloring algorithm has to be very fast. The second, denoted as "start coloring", applies the coloring algorithm to the original subproblem, and the color assigned to each vertex is inherited by that vertex in the set S . The number of colors actually used in S can then be computed to give a (usually weaker) upper bound. The third, denoted as "start and local coloring", is a hybrid. Start coloring and inheriting takes place as in "start coloring", but, if the upper bound obtained does not lead to pruning, then local coloring also takes place. This may give a smaller lower bound that does lead to pruning.
The coloring algorithm used is a simple greedy algorithm. The vertices of the set are colored in turn, and each vertex is colored with the smallest available color. Two variations of this are applied to start coloring only. The saturation degree of a vertex is obtained as follows. Once a new vertex has been colored, the saturation degree of an uncolored vertex is the number of colors in its neighborhood. Vertices are then colored in non-increasing order of saturation degree. This is denoted as "saturation start". The second very similar variation applied to start vertices uses Degree of Saturation (DSatur) coloring [15], itself based on saturation degree, but using a degree ordering to select the first vertex and to break ties. This is denoted as "DSatur start".
Another option that can be used is to order the vertices of each new set S (created as the depth is increased) in non-increasing order of color. This means that, as subsequent vertices of S are considered, the coloring upper bound tends to decrease quickly and proves particularly effective. This option is denoted by "ordered" in the following tables.
There is an important difference between the evaluation here and the general evaluation in [4]. Within HTS,many of the calls to the exact or pseudoexact algorithm supply a good lower bound. Thus, the evaluation here is carried out with the best available lower bound supplied. The time in seconds to complete the algorithms for a number of DIMACS instances [16] used in [4] are shown in Tables 1 and  2. No single algorithm is the best for all instances, but clearly vertex ordering is particularly helpful.

Evaluation of Exact Algorithms with a Coloring Bound for Use in HS2
Although the results in Tables 1 and 2 give a useful comparison, it should be noticed that the subproblems to which exact and pseudoexact search is typically applied in HTS may be much smaller than the instances used in those tables. Thus, four of the algorithms were evaluated further by using them in place of the exact and pseudoexact search in HTS. The critical improvement in performance arises from maximizing the mean number of tabu search iterations per second, and this was calculated for an extended run on each of seven instances. These were DIMACS, BHOSLIB, and permutation code instances used in [2], and, in general, the same parameters were used for HTS as in the experiments in [2]. The results are shown in Table 3.
It appears that there is much less difference in the results than in the previous tables, in part because if the time for exact search is small the time to calculate neighborhoods becomes more significant. However, it appears from further experiments that the third and fourth algorithms (using ordering) allow the parameter Tssetmin in HTS (determining the smallest current clique from which vertices may be removed) to be smaller. These smaller values allow exact search to be applied to larger subproblems without excessively slowing tabu search and can improve performance as a result. From inspection of Table 3, the third algorithm "C&P with ordered local coloring" was selected for use in an improved version HTS2 of HTS. This algorithm also had the merit that it was simpler than the fourth algorithm. Pseudocode for "C&P with ordered local coloring" is shown in Algorithm 1.  if |F| + |S| > max{|Best|, External_lower_bound − 1} then 3: color S greedily, using smallest available color; 4: sort S in non-increasing order of color; 5: select s ∈ S in above order;

Minor Improvements in HTS2
Apart from the change from the exact and pseudoexact search in HTS to the exact "ordered local coloring" algorithm detailed in Algorithm 1, there are some other relatively minor changes from HTS in HTS2. As well as the tabu search algorithm, HTS has a main heuristic that generates good starting solutions (sometimes finds the best solution itself) and i-optimizations for i = 1, 2, 3. These i-optimizations remove i vertices from the current clique in all possible ways and apply the exact algorithm to the neighbor set of vertices adjacent to all remaining vertices in an attempt to find a new larger clique. This continues until there is no improvement. In HTS, there are thresholds θ 1 , θ 2 , θ 3 , θ ts for the four optimizations. In HTS, these thresholds are allowed to be different and are selected so that each optimization is only applied to very promising cases, and 3-optimization never makes HTS unacceptably slow. In HTS2, these four thresholds are replaced by a single threshold θ. The justification for this is that the improved performance of the tabu search makes a different threshold for tabu search unnecessary, and the relative slowness of 3-optimization in some instances can be dealt with by considering the size of the current clique. Thus, the overall structure of HTS2 is as shown in Algorithm 2. C := 2_optimize(C); 13: end if 14: end if 15: if |C| > |BestC| then 16: BestC := C;  [2] is S = [1, 50, 80, 100, 120, 160, 200, 300, 600, 800], and S is fixed to this choice in HTS2 unless the graph has < 1000 vertices when S = [1, 50, 80, 100, 120, 160, 200, 300]. Thus, no decision on an appropriate S is necessary in HTS2. Pseudocode for the main heuristic is presented in Algorithm 3, and pseudocode for the i-optimizations is presented in Algorithm 4.
In the original HTS algorithm a parameter, Nontabu max is used in tabu search. If the neighborhood to which exact search is applied has at least Nontabu max vertices, a protection mechanism is invoked (selecting a random vertex in the neighborhood) to avoid application of the exact search to an excessively large neighborhood. Typical values of 30 or 60 were used. In HTS2, a parameter λ 2 from the main heuristic, to be outlined in the next section, is used instead of Nontabu max.
A final change is to replace the condition that the tabu search optimization runs for Tstime seconds unless the size of the clique increases, by the requirement that it runs for 10,000 iterations, unless the size of the clique increases. This avoids the need for the parameter Tstime and proved satisfactory for all instances. Pseudocode for the revised tabu search algorithm is presented in Algorithm 5.

Simplification of Parameters
There are several parameters used in HTS. These are not difficult to determine by a short exploratory run, but it is more convenient if most of them are determined by the algorithm itself. The parameters in question are θ 1 , θ 2 , θ 3 , θ ts , λ 1 , λ 2 , Tstime, Tstenure, and Tssetmin. Pseudoexact search in HTS also has a number of coefficients, but these are no longer relevant as pseudoexact search is not used in HTS2. The use of Tstime was replaced by a condition "10,000 iterations without improvement" in the previous section.
The four θ parameters in HTS are replaced in HTS2 by a single parameter θ (see Algorithm 2) determined as follows. The main heuristic is run 100 times. If the largest clique C max found is not unique, then θ = |C max |. If this value is unique, then θ is the number of vertices in the second largest clique. This ensures that the other optimizations are only applied to the most promising cliques generated, but avoids single outliers that might only be generated very rarely in later iterations.
The two parameters λ 1 and λ 2 are used in the main heuristic (see Algorithm 3). If the size of the neighborhood N(C) of the current clique satisfies |N(C)| < λ 2 , then the generation of the clique by the main heuristic is completed by applying the exact algorithm to N(C). If the clique generated has at least θ vertices, the algorithm reverts to a somewhat larger stored neighborhood with less than λ 1 vertices and applies the exact algorithm. In HTS2, the two λ parameters are determined by the edge density 2|E|/(|V|(|V| − 1)), as shown in Table 4. The parameter λ 2 is also used in the protection mechanism within tabu search to avoid applying the exact algorithm to excessively large subproblems. Table 4. Choices of parameters λ 1 and λ 2 .
The tabu search parameter Tssetmin is the critical parameter to be set by the user. It must be less than θ, but, if it is too small, attempts may be made to solve exactly subproblems that are too large, requiring the protection mechanism. Otherwise, larger values make tabu search faster, but smaller values may make tabu search more powerful. A balance should be achieved.
It should be noted that the current implementation allows the original θ and λ parameters from HTS to override these choices if the user requires it. Otherwise, Tstenure and Tssetmin are the only relevant parameters in HTS2.

Comparison of HTS, HTS2, and SBTS
This section presents a comparison of the original HTS algorithm, the new HTS2 algorithm, and SBTS. The software for the SBTS algorithm is available online as [17]. The benchmarks selected are BHOSLIB instances [18], some of the harder DIMACS instances [16], Sloane code construction instances [6], and permutation code construction instances from [7]. A few instances shown in [2] to be most easily solved using a pre-processing method are excluded, so the description of pre-processing need not be repeated here. These benchmarks are not ideal in the sense that the optimal solution is known for many of them, but they are at least easily available to allow comparison by others. Using an exact algorithm cannot be expected to be the fastest algorithm, but the more thorough local search might lead to improved solutions if improved solutions are possible. Of particular interest, then, is the permutation code instance 7_5. The vertices correspond to the permutations on 7 symbols, excluding those permutations at Hamming distance 1, 2, 3, or 4 from the identity permutation. Two vertices are adjacent if they are at Hamming distance ≥ 5. A theoretical construction using group theory shows that there exists a clique with 78 vertices [19], but the largest clique found with a maximum clique algorithm previously has 72 vertices (or 73 using the pre-processing method in [2]). For permutation code instances other than 7_4 and 7_5, the vertices of the graph correspond to orbits of permutations under a group, as explained in [7].
The processor used for the experiments was an Intel(R) Core(TM) i3-9100 3.60GHz processor with 4 GB of RAM. In most cases, the parameters used for HTS were the same as given in [2], the parameters for Tstenure and Tssetmin in HTS2 were the same as used for these parameters in HTS in [2], and the default setting for tabu tenure was used in SBTS. Results for BHOSLIB instances are given in Table 5, and results for all other instances are given in Table 6. In these two tables, the first column gives the instance, and the second column gives the number of vertices in the largest known clique (marked with an asterisk if the clique is known to be optimal). The third, fourth, and fifth columns give the time in seconds to find the solution in column 2, with a note of the number of vertices found if the result in column 2 is not achieved.  For most instances, HTS2 is faster than HTS and finds a larger clique for C2000_9 than HTS without pre-processing, and also a larger clique for 7_5 than HTS with or without pre-processing. For BHOSLIB, most DIMACS and Sloane code construction instances, SBTS is much faster than HTS2, as might be expected. However, HTS2 is more competitive on harder DIMACS instances, such as C2000_9, and performs much better than SBTS on 7_5 and 8_5 cyclic. For 7_5, HTS2 found a clique with 75 vertices once and cliques with 74 vertices four times in 364,585 s, whereas SBTS found cliques with 72 vertices twice but could not improve this in 1,399,916.1 s. Similarly, for 8_5 cyclic, HTS2 found a clique with 49 vertices in 6799 s, whereas SBTS found cliques with 46 vertices four times but could not improve this in 61,097.2 s.

Conclusions
The replacement of the pseudoexact algorithm in HTS with an improved exact algorithm in HTS2 has allowed a pure comparison of tabu search using exact search with a more standard type of tabu search. The interest of the authors is in code construction problems, where certain instances only have to be solved once. The critical issue is that the largest clique possible is found, and run time is relatively unimportant. It is clear that SBTS is much faster than HTS2 for most instances and should be used in applications where clique problems have to be solved quickly. HTS2 has found the same size clique in all instances except 7_5 and 8_5 cyclic, where it has found larger cliques with three more vertices. The instance 7_5 is an important addition to the set of standard benchmarks for maximum clique problems, as it is of a similar size to the largest benchmarks, but no algorithm has been able to find a clique with 78 vertices that is known to exist. In summary, HTS2 finds the same size clique as SBTS for 46 instances, more slowly for 40 instances but faster for 4 instances. It finds a larger clique than SBTS for two instances, while SBTS never finds a larger clique than HTS2.

Conflicts of Interest:
The authors declare no conflict of interest.