An Ant Colony Optimization Based on Information Entropy for Constraint Satisfaction Problems

Solving the constraint satisfaction problem (CSP) is to find an assignment of values to variables that satisfies a set of constraints. Ant colony optimization (ACO) is an efficient algorithm for solving CSPs. However, the existing ACO-based algorithms suffer from the constructed assignment with high cost. To improve the solution quality of ACO for solving CSPs, an ant colony optimization based on information entropy (ACOE) is proposed in this paper. The proposed algorithm can automatically call a crossover-based local search according to real-time information entropy. We first describe ACOE for solving CSPs and show how it constructs assignments. Then, we use a ranking-based strategy to update the pheromone, which weights the pheromone according to the rank of these ants. Furthermore, we introduce the crossover-based local search that uses a crossover operation to optimize the current best assignment. Finally, we compare ACOE with seven algorithms on binary CSPs. The experimental results revealed that our method outperformed the other compared algorithms in terms of the cost comparison, data distribution, convergence performance, and hypothesis test.


Introduction
The constraint satisfaction problem (CSP) is an assignment that consists of a set of variables that satisfy some constraints [1][2][3][4]. CSP can be solved by assigning specific values to variables in accordance with the constraint conditions [5][6][7][8]. The problem has been applied in a multitude of domains in real life, such as scheduling [9,10], task planning [11,12], gate assignment [13,14], and the reserve design problem [15,16].
To solve the CSP, complete methods based on the backtracking mechanism [17,18] explore all possible solutions until they find a feasible solution or prove the non-existence of any solution at all. These complete methods are often integrated with filtering technologies, which are effective in the reduction of the domains. Although the completeness appears to be an ideal property, it is difficult to solve high complex CSPs.
As a result, incomplete methods that include pure random walk (PRW) algorithms [5,19] and evolutionary optimization algorithms [20] have been proposed to find the approximately optimal solution in an opportunistic way. The incomplete methods tend to randomly explore the space and follow heuristic means to search for the most promising domains. The efficiency of the PRW algorithms has been already proved by the in-depth studies on some applications of CSPs. However, they could not ensure the quality of solutions as usual. On the premise of guaranteeing the quality of solutions, evolutionary optimization algorithms can greatly improve the search speed.
In recent years, evolutionary optimization algorithms have attracted attention for solving the CSP. EEMDE is a hybrid meta-heuristic differential evolution (DE) algorithm with an element exchange mechanism, and the mechanism uses the individual differential direction of moving length to maintain the population diversity [21]. PS is a discrete particle swarm optimization (PSO) algorithm that uses information about the conflicts between the variables to calculate the velocity of the individual particles [22]. GSABC is a hybrid algorithm combining artificial bee colony (ABC) and greedy local search technology [23]. ACOS [24], ACOD [24], ACON [25] and ACOU [26] are ACO-based algorithms for tackling the problem. ACOS makes use of the smallest-domain-first strategy to reinforce the search process, whereas ACOD applies the dynamic-random strategy to achieve that. ACON takes advantage of a negative-feedback mechanism to make the ant swarm explore the unknown space in the optimization process. ACOU uses a strengthened pheromone updating mechanism to enhance the pheromone on the edge that has never appeared before according to the dynamic information in the optimization process. Among these evolutionary algorithms, the performance of the ACO-based algorithms is superior.
When using the ACO-based algorithms to solve the CSP, the main task is to construct a feasible assignment that does not violate any constraints. Due to a large number of constraints, the feasible assignment is very hard to find in most test cases. Thus, the ACO-based algorithms try to find an assignment with a low cost value as much as possible. Although some ACO-based algorithms have been proven to be able to solve the CSP, they are still easily trapped in a locally optimal state. In this paper, an incomplete method based on ACO is proposed to solve CSPs. The new algorithm is abbreviated as ACOE, which stands for ant colony optimization based on information entropy. The idea of ACOE is that a crossover-based local search (CLS) is automatically called according to the feedback of information entropy.
The contributions of the paper are listed as follows.
(1) A ranking-based pheromone updating strategy is incorporated into the ACOE algorithm to strengthen the exploratory ability of ants. (2) An automatic adjustment mechanism based on information entropy is proposed. By using the mechanism, the proposed algorithm can perform a local search when the algorithm falls into the local optimal state. (3) A crossover-based local search is used in the ACOE algorithm. Through automatically calling the CLS, ACOE is capable of maintaining the diversity of constructed assignments, and accordingly, improve the quality of the assignments.
The remaining parts of this paper are structured as follows. Section 2 gives the definition of CSP and describes the proposed ACOE algorithm for solving the CSP. Section 3 reports and discusses the experimental results. Section 4 draws the conclusion.

Problem Definition
We defined a CSP to be a triple (X, D, C), where X is a finite set of variables, D is a function that associates each variable with its domain, and C is a set of constraints that restrict the values that the variables can assign at the same time. A label <x i , v p > associates variable x i with a value v p from the domain D(x i ). An assignment A is a set of labels where no variables appear more than once. To solve a CSP more conveniently, we represent the CSP (X, D, C) as an undirected graph G = (V, E), where V is the vertex and E is the edge. In the constructed graph, a possible label is represented by a vertex. A path containing <x i , v p > cannot contain another label for variable x i , otherwise a constraint is violated. The cost function of an assignment A, represented by cost(A), is the number of violated constraints in the assignment A. The cost is 0 if the assignment does not violate any constraints.

Original Ant Colony Optimization (ACO)
ACO, proposed by Dorigo et al. [27], solves the optimization problem by simulating the behavior of real ants finding the shortest path between the nest and the food source. The ACO algorithm has the characteristics of distributed computing, information positive feedback, and heuristic search. At present, the algorithm has achieved good results in CSPs.
In ACO, artificial ants live in a discrete world, and their movement is essentially a transition from one discrete state to another. Each artificial ant releases the pheromone after constructing an assignment, and the amount of pheromone released is directly proportional to the quality of the assignment. The probability that the assignment is selected is determined by a probability distribution formula, which is updated by pheromones, heuristic information, and weights. As the probability distribution function is updated, the better assignment will be selected by subsequent ants with a higher probability. At the same time, a small portion of the pheromone is released on each assignment, allowing the ants to try to find assignments that have not been selected before.

Ant Colony Optimization Based on Information Entropy (ACOE)
ACOE follows the basic ACO algorithm for solving CSPs, and the process is shown in Algorithm 1. At each iteration, ant k constructs an assignment Ak. If the cost of Ak is lower than that of the current best assignment bestA, bestA is replaced by Ak; otherwise, bestA is unchanged. Then, the pheromone value on each vertex is updated. The optimization process is repeated until a solution is found by an ant or the maximum number of iterations Nmax is reached. In the following, we first described the assignment construction and the ranking-based pheromone updating. Then, we introduced the automatic adjustment mechanism based on information entropy and the crossover-based local search. Finally, we discussed parameter settings. if the condition is satisfied then 9: bestA ← CLS(bestA) 10: end if 11: end for 12: Update pheromone on each vertex 13: until cost(bestA) = 0 ∨ Nmax is reached

Original Ant Colony Optimization (ACO)
ACO, proposed by Dorigo et al. [27], solves the optimization problem by simulating the behavior of real ants finding the shortest path between the nest and the food source. The ACO algorithm has the characteristics of distributed computing, information positive feedback, and heuristic search. At present, the algorithm has achieved good results in CSPs.
In ACO, artificial ants live in a discrete world, and their movement is essentially a transition from one discrete state to another. Each artificial ant releases the pheromone after constructing an assignment, and the amount of pheromone released is directly proportional to the quality of the assignment. The probability that the assignment is selected is determined by a probability distribution formula, which is updated by pheromones, heuristic information, and weights. As the probability distribution function is updated, the better assignment will be selected by subsequent ants with a higher probability. At the same time, a small portion of the pheromone is released on each assignment, allowing the ants to try to find assignments that have not been selected before.

Ant Colony Optimization Based on Information Entropy (ACOE)
ACOE follows the basic ACO algorithm for solving CSPs, and the process is shown in Algorithm 1. At each iteration, ant k constructs an assignment A k . If the cost of A k is lower than that of the current best assignment bestA, bestA is replaced by A k ; otherwise, bestA is unchanged. Then, the pheromone value on each vertex is updated. The optimization process is repeated until a solution is found by an ant or the maximum number of iterations N max is reached. In the following, we first described the assignment construction and the ranking-based pheromone updating. Then, we introduced the automatic adjustment mechanism based on information entropy and the crossover-based local search. Finally, we discussed parameter settings. Construct a complete assignment A k 5: if cost(A k ) < cost(bestA) then 6: bestA ← A k 7: end if 8: if the condition is satisfied then 9: bestA ← CLS(bestA) 10: end if 11: end for 12: Update pheromone on each vertex 13: until cost(bestA) = 0 ∨ N max is reached 14: return bestA

Assignment Construction
For constructing the assignment, each ant starts with an empty assignment and then iteratively selects the next vertex that is not assigned to the assignment. The probability of selecting the vertex of the assignment A k is defined as: where τ Ak (<x i , v p >) is the pheromone value on the vertex <x i , v p >; α is the parameter determining the weight of the pheromone value; η Ak (<x i , v p >) is the heuristic information of selecting the vertex <x i , v p > [28], which is inversely proportional to the number of new violated constraints when assigning <x i , v p > to A k ; β is the parameter determining the weight of the heuristic information; m is the number of values for each variable. The pseudo-code of the assignment constructed by ant k is given in Algorithm 2.

Ranking-Based Pheromone Updating
After each ant constructs a complete assignment, the pheromone values are updated. In ACOE, the ants are sorted by the costs of the constructed assignments, and the contribution of the pheromone updating is weighted according to the rank r of the ant. We used the weight r for the r-th best ant. Thus, the pheromone values were updated by: where ρ is the pheromone evaporation rate (0 < ρ < 1); r is the ranking index; and ∆τ Ak (<x i , v p >) is the increased pheromone caused by the ant k. If ant k is the r-th best ant, the increased pheromone on the vertex <x i , v p > belonging to the assignment A k is inversely proportional to the cost multiplied by r.
A smaller r causes more pheromones to be increased on the vertices belonging to the assignment. As indicated by (3), the range of τ Ak is between the minimum pheromone τ min and the maximum pheromone τ max in the condition of τ min ≤ τ Ak ≤ τ max (0 < τ min ≤ τ max ) [28]. Once the value of τ Ak exceeds the range, the value will change to the nearest end-point.
The degree to which the global information is contributed depends on the quality of the generated assignments. A better assignment is more likely to make a greater contribution to the future assignments. The pheromone updating strategy based on ant ranking make assignments with lower costs more contribution to the global optimization. Thus, the global search ability of ACOE is enhanced.

Automatic Adjustment Mechanism Based on Information Entropy
Information entropy is used to measure the expected value of a random variable. The larger the information entropy of a variable, the greater its uncertainty, that is, more information is needed to determine this variable. The information entropy of an assignment is the sum of the information entropy of all variables: where p Ak (<x i , v p >) is the probability that the vertex <x i , v p > is selected in the assignment A k ; n is the number of variable; m is the number of value; H(A k ) is the information entropy of the assignment constructed by ant k; and the logarithm takes 2 as the base. ACOE solves the CSP by constantly comparing the current global best assignment and the best assignment in the current iteration. The comparison process for the two assignments is defined in the formula below: where H(BestA) is the current global best assignment; H(Best t A) is the best assignment in the tth iteration; and θ is the specified switch parameter. At the beginning of ACOE, the pheromones on each vertex are equal and the information entropy is the largest. As the number of iterations increases, the pheromones on the vertices that found by ants increase, whereas the pheromones on the other vertices decrease. At the same time, the changing process of these pheromones leads to a reduction in the information entropy of each assignment. When the difference between H(BestA) and H(Best t A) is very small, the proposed algorithm performs a local search (see Section 2.3.4).

Crossover-Based Local Search
To enhance the search ability of ACOE, we incorporated a local search (LS) into the proposed algorithm. The LS uses a crossover operation to optimize the current best assignment. Thus, this LS is called CLS. For solving CSPs, assignments with lower costs are generally more inclined to be selected by ants. Therefore, the excellent assignments with lower costs are selected to explore its neighborhood by using the CLS procedure, and better assignments are expected to be obtained. In ACOE, a crossover operation is performed if the difference of the information entropy between the current global best solution and the best solution in the t th iteration is less than θ. The current best assignment and other randomly selected assignments will be crossed to obtain a new assignment. Suppose the current best where n is the number of variables; the randomly selected assignment is {<x 1 We selected a random integer uniformly distributed between 1 and (n−1) as the crossover point, and we assumed 2 was the intersection point in this example. Then, bestA and A u crossed to generate a new assignment C = {<x 1 , v 1 >, <x 2 , v 2 >, <x 3 , v 4 >, . . . , <x n−1 , v n−2 >, <x n , v n−3 >}. If the newly obtained assignment has a lower cost value than the best assignment, the new assignment will replace the best assignment. Otherwise, the best assignment will be preserved. The pseudo-code of CLS is shown in Algorithm 3.

Algorithm 3 CLS
Input: bestA, number of crossover operations L, number of values m Output: bestA bestA ← C 7: end if 8: end for 9: return bestA 2.3.5. Parameter Setting ACOE has some parameters: The number of ants N ant , the minimum pheromone τ min , the maximum pheromone τ max , the specified switch parameter θ, the pheromone evaporation rate ρ, and the weight parameters α and β. We briefly analyzed the impact of these parameters on this proposed algorithm. N ant was set to 10: The running time will increase if N ant has a larger value; the cost will increase if N ant has a smaller value. τ min was set to 0.01 and τ max was set to = 4 according to previous studies [24,29]. θ was set to 0.01: If θ has a smaller value, CLS can hardly work; if θ has a larger value, the running time will increase due to calling CLS multiple times. β, α, and ρ have an impact on the exploratory behavior of ants. β was set to 10, α was set to 2, and ρ was set to 0.01. ACOE was run 30 times on the same test case (Test 7) with different combinations of β, α, and ρ. Then, the lowest cost value corresponding to a combination of the three parameters was recorded. The details of the experimental results are shown in Table 1. In the table, β was set to 6, 8, and 10; α was set to 2, 3, 4, and 5; ρ was set to 0.01, 0.02, 0.03, 0.04, and 0.05. The other values represent the lowest costs obtained by ACOE with different β, α, and ρ.

Datasets
In the paper, four classes of binary CSP test cases were generated (Class 1, Class 2, Class 3, and Class 4), and each class contained six specific test cases. The generated test cases were represented by four components < n, m, p 1 , p 2 >, where n is the number of variables, m is the domain for each variable, p 1 is the connectivity of the constraint graph, and p 2 is the tightness of the constraints. Furthermore, the constrainedness of a generated test case can be defined by the k-value (the range is 0 to 1), and the k-value can be calculated according to Equation (7) [30,31]. A CSP is under-constrained and can be solved when k is less than 1, whereas a CSP is over-constrained and usually difficult to solve when k is greater than 1. More details of the generated test cases are shown in Table 2. ).

Cost Comparison
The cost value is an important index to evaluate the performance of the compared algorithms. For each test case, we ran eight algorithms (ACOE, ACOS, ACOD, ACON, ACOU, EEMDE, PS, and GSABC) 30 times respectively. The minimum cost (Min), the average cost (Avg), and the maximum cost (Max) were recorded, and the experimental results are given in Table 3. It can be seen from the table that the minimum cost, the average cost, and the maximum cost increase gradually increased with the growth of the k-value. For the small-scale problems with 100 variables (Test 1-20), the minimum cost values obtained by ACOE were not obviously superior to those obtained by the other compared algorithms. The proposed algorithm was not as good as ACON on Test 18, and it was inferior to ACON and ACOU on Test 20. All the average cost values found by ACOE were the lowest, whereas the maximum cost values obtained by this proposed algorithm were the lowest except for Test 18. For the large-scale problems with 150 variables (Test 21-40), ACOE presents more obvious advantages than the other seven compared algorithms. For the 20 test cases, ACOE gets all the best minimum costs, average costs, and maximum costs.

Result Distribution Analysis
In this section, we conducted an analysis about the result distribution of the eight compared algorithms. Test 8, Test 18, Test 28, and Test 38 were selected as the representative of each case. For each representative, all of the cost values obtained by each compared algorithm in 30 runs were used as experimental data. We calculated the minimum point, the first quartile, the median, the third quartile, and the maximum point of the cost values of each test case, and then we used the five statistical quantities to draw the box plots ( Figure 2). The median was used to describe the concentration of the experimental data, regardless of the maximum or minimum value of the data distribution. As can be seen from these box plots, the median cost of ACOE was lower than that of the other compared algorithms. In addition, the length of the box also reflected the concentration of the data. The box length of ACOE was relatively short on the four test cases. The above analysis indicates that the result distribution of the proposed algorithm was the most concentrated.
Entropy 2019, 21, x FOR PEER REVIEW 9 of 15 the data. The box length of ACOE was relatively short on the four test cases. The above analysis indicates that the result distribution of the proposed algorithm was the most concentrated.

Convergence Analysis
Convergence means that the objective cost value evaluated by an algorithm tends to be stable after several iterations. We compared the convergence of the eight compared algorithms on Test

Convergence Analysis
Convergence means that the objective cost value evaluated by an algorithm tends to be stable after several iterations. We compared the convergence of the eight compared algorithms on Test

Hypothesis Test
For the purpose of proving the validity of ACOE in coping with CSPs, we used the fisher-indep hypothesis test with a 0.05 confidence level. Thus, a significant difference could be reflected between two algorithms when the p-value was below 0.05. All the cost values obtained by each compared algorithm in 30 runs were collected as experimental data. The comparative results of ACOE with ACOS, ACOD, ACON, ACOU, EEMDE, PS, and GSABC are shown in Table 4. For some small-scale test cases, such as Test 1 and Test 2, ACOE was not significantly different from the other seven compared algorithms. On Test 3 and Test 11, the proposed algorithm was only significantly different from PS. ACOE performed significantly than ACOS and PS on Test 4, and it performed significantly than EEMDE, PS, and GSABC on Test 12, Test 18, and Test 20. On Test 6 and Test 13, the proposed algorithm did not perform significantly better than ACON and ACOU. The proposed algorithm was not significantly better than ACOD and ACOU on Test 7, and it was not significantly better than ACON on Test 9 and Test 19. On the large-scale test cases (Test 21-40), the p-values were less than 0.05 when ACOE was compared to ACOS, ACOD, ACON, ACOU, EEMDE, PS, and GSABC, which indicates ACOE was significantly better than the other algorithms.

Hypothesis Test
For the purpose of proving the validity of ACOE in coping with CSPs, we used the fisher-indep hypothesis test with a 0.05 confidence level. Thus, a significant difference could be reflected between two algorithms when the p-value was below 0.05. All the cost values obtained by each compared algorithm in 30 runs were collected as experimental data. The comparative results of ACOE with ACOS, ACOD, ACON, ACOU, EEMDE, PS, and GSABC are shown in Table 4. For some small-scale test cases, such as Test 1 and Test 2, ACOE was not significantly different from the other seven compared algorithms. On Test 3 and Test 11, the proposed algorithm was only significantly different from PS. ACOE performed significantly than ACOS and PS on Test 4, and it performed significantly than EEMDE, PS, and GSABC on Test 12, Test 18, and Test 20. On Test 6 and Test 13, the proposed algorithm did not perform significantly better than ACON and ACOU. The proposed algorithm was not significantly better than ACOD and ACOU on Test 7, and it was not significantly better than ACON on Test 9 and Test 19. On the large-scale test cases (Test 21-40), the p-values were less than 0.05 when ACOE was compared to ACOS, ACOD, ACON, ACOU, EEMDE, PS, and GSABC, which indicates ACOE was significantly better than the other algorithms.

Conclusions
CSP, as a topic of artificial intelligence, plays an important role in many real-life applications. In the paper, the ACOE algorithm was proposed to deal with the problem. On the generated CSP test cases, the performance of ACOE was evaluated from the aspects of cost comparison, data distribution, convergence performance, and hypothesis test. The results showed that the proposed algorithm had advantages in efficiency and effectiveness. However, there were limitations about the proposed algorithm on the next two aspects. First of all, although we had introduced some measurements to evaluate the performance of different algorithms, it is worth mentioning here that other evaluation measures like running time and standard deviation could be also be applied for a wider range of performance analysis. Secondly, although ACOE was evaluated on 40 test cases, the algorithm was not tested on real datasets. In the future, we will focus on the application of the proposed algorithm on real data.