A Hybrid Imperialist Competitive Algorithm for the Distributed Unrelated Parallel Machines Scheduling Problem

: In this paper, the distributed unrelated parallel machines scheduling problem (DUPMSP) is studied and a hybrid imperialist competitive algorithm (HICA) is proposed to minimize total tardiness. All empires were categorized into three types: the strongest empire, the weakest empire, and other empires; the diversiﬁed assimilation was implemented by using different search operator in the different types of empires, and a novel imperialist competition was implemented among all empires except the strongest one. The knowledge-based local search was embedded. Extensive experiments were conducted to compare the HICA with other algorithms from the literature. The computational results demonstrated that new strategies were effective and the HICA is a promising approach to solving the DUPMSP.


Introduction
In recent years, single-factory or centralized production has been continuously replaced by multi-factory production or distributed manufacturing with the further development of globalization. Distributed manufacturing enables manufacturers to be closer to their customers and suppliers, to produce and market their products more effectively, to respond to market changes more quickly and achieve better product quality, lower production cost, reduce management risk, etc. As an important part of manufacturing systems, scheduling is shifted from single-factory scheduling to distributed scheduling with the change of production mode. Distributed scheduling has been considered fully in the past decade .
As a common distributed scheduling problem, the distributed parallel machine scheduling problem (DPMSP) has attracted some attention since the works of Hooker [1]. Chen and Pundoor [2] analyzed the computational complexity of scheduling in supply chains and proposed some fast heuristics. Terrazas-Moreno and Grossmann [3] gave a hybrid bi-level and spatial Lagrangian decomposition method for the scheduling and planning problem in multiple factories with different sites. Behnamian and Fatemi Ghomi [4] applied a heuristic and a genetic algorithm (GA) with a new encoding scheme to minimize the makespan in heterogeneous factories. Behnamian [5] provided a decomposition-based hybrid variable neighborhood search/tabu search algorithm to deal with a DPMSP with the total cost of some factories and the profit of other factories. Behnamian and Fatemi Ghomi [6] developed a Monte-Carlo-based heuristic with seven local search algorithms to minimize the cost-related objective. Behnamian [7] designed a two-phase algorithm with particle swarm optimization for a DPMSP with parallel jobs.
Lei et al. [8] dealt with the distributed unrelated parallel machine scheduling problem (DUPMSP) in heterogeneous factories and proposed a novel imperialist competitive algorithm (ICA) with memory, new revolution, and imperialist competition to minimize the makespan. Lei and Liu [9] studied the DUPMSP with preventive maintenance and makespan minimization and presented an artificial bee colony algorithm in which the wholes swarm is divided into one employed bee colony and three onlooker bee colonies, and the four colonies differed from each other in their search strategies. Lei et al. [10] considered a multi-objective DUPMSP and developed an improved ABC algorithm to minimize the makespan and total tardiness simultaneously, in which problem-related knowledge was proven and knowledge-based neighborhood search was proposed. Pan et al. [11] solved an energy-efficient DUPMSP by using a knowledge-based two-population optimization algorithm to minimize total energy consumption and total tardiness simultaneously, in which the nondominated sorting genetic algorithm-II and differential evolution performed cooperatively and two knowledge-based local search operators were proposed.
The ICA is constructed by simulating sociopolitical behaviors. It is made up of initialization, initial empires, assimilation, revolution, and imperialist competition. Unlike other meta-heuristics [31][32][33], the ICA has some characteristic features, which are a good neighborhood search ability, an effective global search property, and a good convergence rate [34].
As an effective method for solving production scheduling [35], the ICA has been extensively applied to many scheduling problems. Banisadr et al. [36] gave a hybrid ICA for single-machine scheduling. For assembly flow shop scheduling, Shokrollahpour et al. [37] developed a new ICA to optimize two conflicting objectives, and Seidgar et al. [38] proposed an ICA to solve a two-stage problem. With respect to the flexible job shop scheduling problem, Zandieh et al. [39] applied an improved ICA to solve the problem with maintenance. Karimi et al. [40] developed a hybrid algorithm with the ICA and simulated annealing for a problem with transportation times. Lei et al. [41] presented a two-phase algorithm based on the ICA and variable neighborhood search for a problem with an energy consumption threshold. Abedi et al. [42] gave a multi-objective ICA for identical parallel batch-processing machines' scheduling. Yazdani et al. [43] proposed a hybrid algorithm with the ICA and local search for a two-agent parallel machine scheduling problem. An improved ICA was proposed by Zhang et al. [44] to solve photolithography machines' scheduling. Li et al. [45] presented an ICA with empire cooperation for solving a fuzzy distributed assembly scheduling problem. Li et al. [46] proposed an ICA with feedback to solve an energy-efficient FJSP with transportation and a sequence-dependent setup time.
As a meta-heuristic with significant features, the ICA has great potential to generate high-quality solutions for the DPMSP; on the other hand, in the existing ICA [34][35][36], each empire has the same search operator in assimilation, all empires compete in imperialist competition, and the problem-related properties are not used fully. In general, assimilation and revolution should be performed based on the features of the empires. Generally, assimilation in the weakest empire should be intensified and implemented by moving colonies toward imperialists of other empires or other good solutions. It is useful to avoid prematurely excluding the strongest empire from imperialist competition, and the effective usage of problem-related properties or knowledge can improve the search efficiency. Thus, it is beneficial to investigate these improvements of the ICA to solve the DUPMSP efficiently.
In this study, the DUPMSP with total tardiness was considered and a hybrid imperialist competitive algorithm (HICA) is proposed. In the HICA, all empires are divided into three types: the strongest empire, the weakest empire, and other empires. Diversified assimilation was implemented, that is each kind of empire was provided a different assimilation from other kinds of empires. A novel imperialist competition was implemented among all empires except the first type of empire. Problem-related knowledge was included in the local search, and an effective way was applied to integrate knowledge with the ICA. Many computational experiments were conducted. The computational results validated the effectiveness of the new strategies of the HICA and its promising advantages in the DUPMSP.
The paper is organized as follows. The problem description is introduced in Section 2. The HICA for the DUPMSP is described in Section 3. Numerical test experiments are given in Section 4, and the conclusions and the future topics are provided in the final section.

Problem Description
The DUPMSP is described below. There exist n jobs J 1 , J 2 , · · · , J n processed among F factories at different sites. Factory f = 1, 2, · · · , F possesses m f unrelated parallel machines, which are M s f +1 , · · · , M s f +m f and available at all times, where The total number of machines in all factories is W = ∑ F f =1 m f . Each job J i is available at Time 0 and has a due date d i and p ik , which is the processing time of job J i on machine M k , k = 1, 2, · · · , W.
There are some constraints on jobs and machines. Each machine can deal with at most one job at a time.
No jobs can be processed on more than one machine at a time.
Operations are not allowed to be interrupted. The goal of the problem is to minimize the total tardiness.
where C i and T i are the completion time and tardiness of job J i , T i = max{C i − d i , 0}, and Tard tot indicates the total tardiness. The DUPMSP is the extended version of the unrelated parallel machine scheduling problem (PMSP). The PMSP with total tardiness is NP-hard, so the DUPMSP with total tardiness is also NP-hard. When all parallel machines are identical and the objective is the makespan, any two machines are symmetrical to each other, for example for M s f +1 and M s f +2 , a job can be finished at the same time when it is processed on these two machines, respectively. In the DUPMSP, there is a certain amount of destruction of the above symmetry; however, the symmetry still exists.
For the DUPMSP with total tardiness, it consists of three sub-problems: factory assignment, machine assignment, and scheduling. Factory assignment is used to decide which jobs are allocated into each factory, and machine assignment is for selecting an appropriate machine for each job in a factory. Lei et al. [8] analyzed the strong coupled relation among sub-problems and pointed out that two assignment sub-problems can be integrated into an extended machine assignment, in which each job is allocated to one of W machines, not limited to a machine in a factory.

HICA for the DUPMSP
In ICA, a country represents a solution of the problem, and solutions in population P are categorized into two parts: imperialists and colonies, the former are some best solutions in P and the latter are all solutions of P except imperialists. The main steps of the ICA consist of the initial empires' construction, assimilation, revolution, and imperialist competition.
In the existing ICA [34,36,41], the improvements are made by using new strategies on initial empires, assimilation, revolution, and imperialist competition; however, assimilation and revolution are frequently executed in the same ways for all empires and seldom implemented in terms of the characteristics of empires; on the other hand, if the strongest empire has the biggest normalized total cost TC k , then this empire wins likely in the imperialist competition, and colonies can be easily reallocated into the empire; as a result, premature removal may occur. A novel algorithm named the HICA is designed based on the above analyses.

Initialization and Initial Empires
As stated above, the DUPMSP is composed of an extended machine assignment and scheduling. Lei et al. [10] proposed a two-string representation. For the DUPMSP, its solution is represented as a machine assignment string [M h 1 , M h 2 , · · · , M h n ] and a scheduling string [q 1 , q 2 , · · · , q n ]. Machine M h i is allocated for job J i , and q l is a real number and corresponds to J l . The decoding procedure [10] was directly used in this study. Initial population P with N solutions is randomly generated, then all solutions are sorted according to the cost. N im solutions with the smallest cost are chosen as imperialists, and the remaining countries are used as colonies, then the initial empires are constructed. Algorithm 1 shows the steps of the initial empires, where Tard tot,i is the total tardiness of the i-th imperialist, and F k and NC k are defined by: NC k is the number of colonies possessed by imperialist k. S im is the set of all imperialists. N col = N − N im indicates the number of colonies. round(x) denotes the nearest integer to x. Algorithm 1 Initial empires.
1: Determine N im imperialists from population P, and sort them in ascending order of total tardiness 2: Compute the normalized costc k = Tard tot,N im −k+1 for imperialist k = 1, 2, · · · , N im 3: Calculate the power F k and NC k and randomly allocate NC k colonies for each imperialist In the HICA, the cost c i of a solution x i is defined as its objective value. When all imperialists are sorted, Imperialist 1 has the smallest total tardiness Tard 1,tot , Imperialist 2 possesses the second smallest total tardiness Tard 2,tot , and so on; obviously, Tard 1,tot ≤ Tard 2,tot ≤ · · · ≤ Tard N im ,tot .
For each initial empire k, its total cost TC k is computed. Suppose that TC 1 ≤ TC 2 ≤ · · · ≤ TC N im , that is Empire 1 is the strongest one and empire N im is the weakest one. All empires are divided into three types: the first type just has Empire 1, and third type only has empire N im ; the second type is made up of empires 2, · · · , N im − 1.
where Q k is the set of colonies possessed by imperialist k and ζ is a real number and set to 0.1.

Diversified Assimilation and New Revolution
Assimilation and revolution are the main paths to produce new solutions. In general, the process of assimilation is identical for all empires. In the HICA, diversified assimilation is implemented, that is assimilation is performed differently in the different kinds of empires. Algorithm 2 shows the steps of diversified assimilation, where α is a real number, β is an integer, and the retained set Ω is used to store the solutions generated in Empire 1, that is a new solution is produced in Empire 1 and added into Ω directly, U A i and U R i are the probability of assimilation and revolution. In this study, we set Assimilation is executed by a global search between the colony and its learning object. In all empires except the worst one, the learning objective of the colony is its imperialist. For colony x i and its learning object y, the global search is shown below. Two positions k 1 , k 2 are stochastically decided, and machines between these positions on the first string of x i are directly replaced with those on the first string of y; a new solution z is generated; if z has smaller Tard tot than or identical Tard tot with x i , then replace x i with z; otherwise, genes of z between two positions k 1 and k 2 of the scheduling string are displaced by those of y on the same positions; if the newly obtained solution z and x i meets the above condition, then z substitutes for x i .

Algorithm 2 Diversified assimilation and new revolution.
1: Add all solutions of Empire 1 into the retained set Ω 2: Choose colonies from Empire 1 by probability U A 1 , and execute assimilation between each chosen colony and its imperialist. 3: Select colonies of Empire 1 in terms of revolution probability U R 2 , and apply multiple neighborhood search to each chosen colony. 4: Choose the best NC 1 + 1 solutions from the retained set Ω; replace all solutions in Empire 1; decide on the new imperialist. 5: for k = 2 to N im − 1 do 6: ifc k < α × c k then 8: implement assimilation with U A 1 and revolution with U R 1 as done in Empire 1. end if 12: end for 13: For empire N im , choose colonies with probability U A 2 ; execute assimilation between each chosen colony and one of the best β solutions in Ω; perform revolution with U R 3 in the same way as for Empire 1.
In Algorithm 2, the assimilation probability or learning object differ from each other in different types of empires. For example, in the worst empire, its imperialist may have low quality and cannot guide the colony, so the colony learns from one best solution of P to obtain high-quality solutions.
Algorithm 2 also shows revolution in each type of empire. A multiple-neighborhood search is used in revolution. There are seven neighborhood structures: N 1 , N 2 , · · · , N 7 . N 1 is described as follows. Move a job with the biggest tardiness from its current machine M k to a randomly chosen machine M w (w = k). N 2 is performed for two randomly chosen machines M k 1 and M k 2 ; shift the job with the biggest tardiness from M k 1 to M k 2 . N 3 is also applied for two randomly chosen machines M k 1 and M k 2 by exchanging the job with the biggest tardiness on M k 1 and the job with the smallest tardiness on M k 2 . N 4 acts on the scheduling string by exchanging two randomly chosen genes q i 1 and q i 2 . N 5 is used to produce a new solution by inserting a randomly chosen gene q i into a new position l, l = i. N 6 is shown as follows. Two positions k 1 and k 2 are randomly chosen, and genes between two positions are reversed. N 7 is depicted below. Begin with machine M k with the biggest total tardiness of jobs; all jobs on M k are in the ascending order of their due dates, then let the sequence of their q l of these jobs be identical to the order of the due dates. For example, for jobs J 1 , J 17 , J 11 on machine M 1 , the ascending order of their due dates is J 11 , J 17 , J 1 , and the sequence of their q l becomes 0.33, 0.23, 0.32, that is new q 1 is 0.33, new q 11 = 0.23, and new q 17 = 0.32.
The revolution of colony x i is shown as follows. Let g = 1, t = 1; repeat the following steps until t = R: a new solution z ∈ N g (x i ) is produced; if x i can be replaced with z according to the condition in the assimilation, then z substitutes for x i ; otherwise, g = g + 1; When there are only two empires, colonies are chosen in terms of U A 1 for assimilation between the colony and its imperialist, and revolution is implemented using U R 1 and the above multiple-neighborhood search.
When assimilation and revolution are performed in all empires, the exchange step is executed. In each empire k, if the total tardiness of imperialist k is bigger than that of a colony, then imperialist k is exchanged with the colony.
Total cost TC k is calculated for each empire k, and all empires are categorized into three types, as done in Section 3.1.

Imperialist Competition
In general, all empires compete with each other, and the weakest colony of the weakest empire is directly added into the winning empire; in this study, a new imperialist competition was applied and shown in Algorithm 3, where TC k is defined by: When N im ≤ 2, all empires compete with each other, as done in the above procedure.
1: Determine the normalized total cost TC k for empire k = 2, 3, · · · , N im 2: Compute power POW k for empire k = 2, · · · , N im and construct vector [POW 2 − r 2 , · · · , POW N im − r N im ] 3: Decide on an empire g with the biggest POW g − r g 4: Randomly choose a solution x from Empire 1; directly substitute for the weakest colony; add into empire g; then, execute multiple-neighborhood search on x, as done in revolution.
Unlike the existing imperialist competition, our competition process directly eliminates the possibility of including the weakest colony in empire g because the weakest colony is a worse solution and difficult to improve even if it is included in a strong empire.

Local Search
Problem-related knowledge was incorporated in the search procedure of the scheduling algorithm [10,11,[47][48][49]. The inclusion of knowledge can avoid useless searching and make the algorithm search in the possible regions of the optimal solutions. There are some papers on knowledge-based scheduling [47][48][49], and the knowledge used is mainly about the makespan. In this study, the knowledge-based local search acts on the imperialist of each empire to intensify the local search ability of the HICA.
Theorem [10]: For two adjacent jobs J i and J j on a machine M k , suppose t is the beginning time of J i : (1) If T i > 0 and T j > 0: a. If t + p jk − d j ≤ 0 and t + p ik − d j > 0, then the sum of their tardiness will diminish after two jobs are exchanged; b. If t + p jk − d j > 0 and p ik − p jk > 0, then the sum of their tardiness will diminish after two jobs are exchanged; (2) If T i ≤ 0, T j > 0, and one of the following conditions is met: then the sum of their tardiness will diminish after two jobs are exchanged.
T * i and T * j are the tardiness of J i and J j after the exchange. The local search is shown below. For empire k = 1, 2, · · · , N im , the following steps are executed on the imperialist of the empire: on each machine M l , l = 1, 2, · · · , W, start with the first job on M l ; repeat the following steps until all adjacent jobs are checked: for Job J i and its adjacent job J j , exchange them if they meet the conditions in the theorem.

Algorithm Description
The HICA is shown in Algorithm 4, and its flow chart is described in Figure 1.

Algorithm 4 HICA.
1: Randomly produce an initial population P 2: while stopping condition is not met do 3: Construct initial empires 4: if N im > 2 then 5: Sort empires, and categorize empires into three types 6: Execute Algorithm 2, and exchange for each empire 7: Implement the new imperialist competition for all empires except Empire 1 8: else 9: Perform Algorithm 2, and exchange 10: Execute imperialist competition 11: end if 12: Apply local search to the imperialist of each empire 13: end while (3) The knowledge-based local search is used to improve the imperialists of all empires for high search efficiency. The above features can provide a good balance between exploration and exploitation for the HICA and result in good performance.

Computational Experiments
Extensive experiments were conducted to test the performance of the HICA for the DUPMSP with total tardiness. All experiments were implemented by using Microsoft Visual C++ 2015 and run on a 4.0 G RAM 2.00 GHz CPU PC.

Test Instances, Metrics, and Comparative Algorithms
Ninety-six combinations were randomly generated, and their basic descriptions are given in Table 1. Five instances were stochastically produced for each combination, and four-hundred eighty instances were obtained. The due date of J i is decided by:  [50]) algorithm for a non-identical PMSP with total weighted tardiness. This GA can be directly applied to solve the DUPMSP. This PSO algorithm also can be used to solve our DUPMSP after the decoding part is replaced with the decoding procedure in the HICA, so they were chosen as comparative algorithm.
Each algorithm ran 20 times for each instance. For a combination, mn i and mx i were the best solution, the worst solution obtained in 20 runs for its instance i = 1, 2, 3, 4, 5, respectively. ag i is the average makespan of 20 elite solutions for instance i. Three indices Min, Max, and Avg are defined by Min = ∑ 5 i=1 mn i /5, Max = ∑ 5 i=1 mx i /5, and Avg = ∑ 5 i=1 ag i /5.

Parameter Settings
The HICA has the following parameters: N, N im , max_it, U A 1 , U R 1 . The Taguchi method is often applied to decide parameter settings for optimization algorithm and was also adopted in this study. Table 2 describes the levels of all parameters. The HICA ran 20 times for an instance of Problem Combination 14. the orthogonal array L 27 (3 5 ) was executed. The results of the Min and S/N ratio are shown in Figure 2. The S/N ratio is defined as −10 log 10 0.1 × Min 2 , where Min denotes the best solution obtained in 20 runs. As shown in Figure 2, the best settings were N = 100, N im = 6, max_it = 10 5 , U A 1 = 0.7, U R 1 = 0.3.

Results and Discussion
Two variants named HICA1 and HICA2 are given. HICA1 was obtained after the knowledge-based local search was deleted from the HICA. The comparison between the HICA and HICA1 was to validate the impact of the local search on the performance of the HICA. In HICA2, when assimilation was performed, each colony moved toward its imperialist. The usage of HICA2 was to show if the diversified assimilation was useful to improve the performance of the HICA.
Tables 3-6 describe the computational results of the HICA, the two variants, and the two comparative algorithms, in which data highlighted in bold are results of HICA being better than any other algorithms. The Wilcoxon test was performed, and the corresponding results are shown in Table 7. The box plots of all algorithms are given in          It can be found from Tables 3-6 that the HICA performed better than its two variants on the three metrics. The HICA produced better Min than HICA1 on 65 combinations; the Max of HICA1 was less than that of the HICA on 37 combinations; the HICA had a smaller Avg than HICA1 on 69 combinations. Obviously, the HICA had better convergence performance than HICA1, and the inclusion of the local search really improved the performance of the HICA. HICA2 could not generate better results than the HICA on most of the instances and was superior to the HICA on a limited number of instances. In the HICA, assimilation was implemented differently in different types of empires. The comparison between the HICA and HICA2 really proved the effectiveness of the diversified assimilation. Table 7 reveals that the HICA had a better Min, Max, and Avg than HICA1 and HICA2 in the statistical sense and produced a similar Std as its two variants; this conclusion also can be seen from  The parameters of the GA and the PSO algorithm were directly adopted from Behnamian and Fatemi Ghomi [4] and Hulett et al. [50], except the stopping condition. Three algorithms had the same termination condition: max_it = 10 5 . The computational results and times are shown in Tables 3-6 and 8, respectively. As shown in Tables 3-7, the HICA generated a smaller Min than the two comparative algorithms on all instances, that is the HICA converged significantly better than the PSO algorithm and the GA. The Max and Avg of the HICA were less than those of the PSO algorithm on 95 of 96 combinations and better than those of the GA on all combinations.
The HICA obtained a smaller Std than the GA on most of the instances. The performance differences on the Max, Min, and Avg with respect to the HICA, PSO algorithm, and GA also can be found from  Although HICA performed worse than the PSO algorithm on the Std, the HICA possessed better convergence, smaller average results, and a smaller Max than its two comparative algorithms.
In the HICA, The strongest empire was excluded from imperialist competition to avoid premature removal. The diversified assimilation and new revolution were implemented differently in the different types of empires, and the local search could effectively improve the search efficiency; on the contrary, the PSO algorithm and GA have a strong global search ability; however, their local search ability was not intensified, and this feature was the main reason for their low performance. Thus, the HICA can effectively solve the DUPMSP.

Conclusions
In this study, a new algorithm by hybridizing the ICA with the knowledge-based local search was proposed to solve the DUPMSP with total tardiness minimization, in which empires were divided into three types. To obtain high-quality solutions, the diversified assimilation and new revolution were designed; imperialist competition was newly implemented on N im − 1 empires without the strongest one; the problem-related properties were proven; the knowledge-based local search was applied to improve the quality of the imperialists. Extensive experiments were conducted on 480 instances. The computational results demonstrated that the new strategies were effective and that the HICA had promising advantages in the considered DUPMSP.
The DPMSP is an important scheduling topic. We will focus on the DPMSP with various constraints such as additional resources and machine eligibility and try to solve the problem by using meta-heuristics with new optimization mechanisms such as reinforcement learning. Other distributed scheduling problems including distributed assembly hybrid flow shop scheduling are also our future topics. We will solve distributed scheduling problems with factory eligibility or energy-related constraints by using reinforcementlearning-based meta-heuristics.