An Artiﬁcial Bee Colony with Adaptive Competition for the Unrelated Parallel Machine Scheduling Problem with Additional Resources and Maintenance

: The unrelated parallel machine scheduling problem (UPMSP) is a typical production scheduling problem with certain symmetries on machines. Additional resources and preventive maintenance (PM) extensively exist on parallel machines; however, UPMSP with additional resources and PM has been scarcely investigated. Adaptive competition is also seldom implemented in the artiﬁcial bee colony algorithm for production scheduling. In this study, UPMSP with additional resources and PM is investigated, which has certain symmetries with machines. An artiﬁcial bee colony with adaptive competition (ABC-AC) is proposed to minimize the makespan. Two employed bee swarms are constructed and evaluated. In the employed bee phase, adaptive competition is used to dynamically decide two cases. The ﬁrst is the shifting of search resources from the employed bee swarm with a lower evolution quality to another one, and the second is the migration of solutions from the employed bee swarm with a higher evolution quality to another one. An adaptive onlooker bee phase and a new scout phase are given. Extensive experiments are conducted on 300 instances. The computational results demonstrate that the new strategies of ABC-AC are effective, and ABC-AC provides promising results for the considered UPMSP.


Introduction
In the past decades, UPMSP has been extensively considered, and in most of works on UPMSP, the machine is the only considered resource; however, additional resources often exist in many real-world parallel machine manufacturing process. Additional resources include automated guided vehicles, machine operators, tools, pallets, dies and industrial robots, and the total number of the used additional resources on each machine cannot exceed a given threshold at any time. Unrelated parallel machine scheduling problem with additional resources (UPMSPR) has become a significant area of scheduling research, and many results have been obtained [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16].
There are two types of UPMSPR. The first is UPMSPR with one additional resource [2][3][4][5][6][7][8]. Zheng and Wang [3] proposed a two-stage adaptive fruit fly optimization algorithm (TAFOA) with a heuristic and knowledge-guided search for the problem with renewable resource. Fanjul-Peyro et al. [4] presented two integer linear programming models and three matheuristics. Fleszar and Hindi [5] presented an efficient mixed-integer linear programming (MILP) model and a constraint programming model.
UPMSP and UPMSPR with PM that ABC is a potential good optimization algorithm to solve UPMSPR with PM; therefore, ABC is used.
When two bee swarms or multiple bee swarms are used, adaptive competition as an effective way is implemented among bee swarms in previous ABC [43,44]. Chu et al. [44] presented an adaptive competition by evaluation of swarm and solutions of migration from the inferior swarm to superior one. Wang et al. [43] proposed an adaptive competition in the onlooker bee phase, in which two employed bee swarms are given a selection probability. Adaptive competitions can effectively use search advantages of the winning bee swarms and intensify the search efficiency of ABC; however, adaptive competition is seldom investigated in ABC, and the corresponding implementations are also limited.
In this study, UPMSPR with PM and makespan minimization is considered. An effective way is provided to implement adaptive competition, and a novel artificial bee colony with adaptive competition (ABC-AC) is proposed. Two employed bee swarms are constructed and compared according to evolution quality. In the employed bee phase, adaptive competition is fulfilled to dynamically select the following two cases. The first is the shifting of search resources from the employed bee swarm with lower evolution quality to another one, and the second is the migration of solutions from the employed bee swarm with higher evolution quality to another one. An adaptive onlooker bee phase is implemented, and a new scout phase is given. A number of experiments are conducted. The computational results demonstrate that new strategies of ABC-AC are effective and that ABC-AC is a competitive algorithm for solving the considered UPMSPR.
The rest of the paper is arranged as follows. Section 2 describes the considered UPMSPR with PM. ABC is introduced in Section 3. Section 4 shows the detailed steps of ABC-AC for UPMSPR with PM. Section 5 presents the computational results and analyses. The conclusions and future topics are reported in the final section.

Problem Description
UPMSPR with PM is described as follows. There are n jobs J 1 , J 2 , · · · , J n and m unrelated parallel machines M 1 , M 2 , · · · , M m . Each job can be processed on any one of m machines. p ki indicates processing time of job J i on machine M k . An additional renewable resource is considered. Job J i processed on M k needs r ki units of the additional resource. At most, R max units of the additional resource can be used at any time.
To keep the manufacturing system at the desired level of operation, PM is considered. A time interval exists between two consecutive PM and jobs are processed in the interval. u k is the length of the interval on M k , w k indicates the duration of PM on M k and the beginning time of the g-th PM is g × u k .
There have following constraints on jobs and machines. All jobs and machines are available at time zero. Each job can be processed on only one machine at a time.
Each machine handles at most one job at a time. Preemption is not allowed. The problem is composed of the scheduling sub-problem and machine assignment sub-problem. The goal of the problem is to minimize the makespan.
where C max indicates the maximum completion time of all jobs. For UPMSP with the objective of makespan, on each machine, there exists a job-related symmetry, that is, two adjacent jobs are exchanged, and the objective is not changed. When additional resources are considered, there is a certain amount of destruction on the above symmetry; however, the symmetry still exists.

Introduction to ABC
In ABC, there are three types of artificial bees. The first is employed bee, who searches for the food source. The second is the onlooker bee, who is in the hive to choose a food source. The third is the scout, who does random searches for a new food source. A solution of the problem is depicted as the position of a food source, the nectar amount of which is the fitness of the solution.
In the search process, the initial population P with N solutions is first produced, and then three phases-bee phase, onlooker bee phase and scout phase-are performed repeatedly before the stopping condition is met.
In the employed bee phase, a new solution y i is produced for each x i ∈ P.
where φ ∈ [−1, 1] is a real random number, and x k ∈ P is a randomly selected solution, i = k. Greedy selection is applied between x i and y i : if f it(y i ) > f it(x i ) , then y i substitutes for x i , where f it(x i ) denotes the fitness of x i .
In the onlooker bee phase, each onlooker bee chooses a food source by roulette selection based on the probability defined by where prob i indicates the probability of solution x i . Once an onlooker bee selects a food solution x i , a new solution y i is obtained by Equation (2) and the above greedy selection is applied to decide if x i can be replaced with y i .
In the above two phases, a trial i is computed for each x i . Initially, trial i = 0 for all solutions in P. If the newly obtained y i cannot update x i , then trial i = trial i + 1; otherwise, In scout phase, if trial i of a food source exceeds a threshold Limit, the corresponding employed bee will turn into a scout, which randomly produces a food source to substitute for the old one.

ABC-AC for UPMSPR with PM
Competition is often performed among populations or swarms in the following way. After populations or swarms are evaluated, the winning population or swarm are determined, and then solutions of other population or swarm are migrated to the winning one. In this study, a new way is applied to execute adaptive competition, in which solution migration or search resource shifting between two employed bee swarms are dynamically decided according to competition results, adaptive onlooker bee phase and a scout phase are also newly implemented. The detailed descriptions are shown below.

Solution Representation
In this study, a new solution representation is presented. For UPMSPR with n jobs, m machines, R max units of the additional resource and PM, its solution is represented as a machine assignment string [M h 1 , M h 2 , · · · , M h n ] and a scheduling string [θ 1 , θ 2 , · · · , θ n ], where M h i is the assigned machine for job J i and θ i is real number.
The decoding procedure is shown below.
(1) All assigned jobs on each machine M k , k = 1, 2, · · · , M m are decided in terms of machine assignment string; (2) For each machine M k , k = 1, 2, · · · , m, (1) a permutation of all jobs on M k is gotten by sorting these jobs in the ascending order of θ i , (2) for the permutation, start with first job, for each job J i , first decide each idle period of M k , if J i can be inserted into some idle periods when processing time and resource constraint are met, then choose an idle period with the smallest beginning time and insert J i into the chosen period in terms of processing time and resource constraint; otherwise, J i is processed after the current last processed job of M k ; if the completion time of J i exceeds g × u k , then PM is first done, and then J i is processed.
All constraints of UPMSPR with PM are directly handled in the decoding procedure, and the obtained schedule is always feasible. A solution of the example in Section 2 is [2, 2, 1, 1, 1, 1, 1, 2] and [0.22, 0.72, 0.11, 0.84, 0.03, 0.35, 0.52, 0.17], and the corresponding schedule is depicted in Figure 1. As shown in Figure 1, J 5 is assigned on M 1 , s 5 = 0 and C 5 = 4; moreover, all constraints are met, and thus a feasible schedule is obtained.
A solution with m job sequences [3] and a representation method with three strings [11] are used to denote solutions of UPMSPR; however, strings or job sequences in these methods are dependent each other. In this study, machine assignment string and scheduling string are independent, and additional resources are effectively handled in the decoding procedure.
The initial population with N solutions is produced as follows. A heuristic is presented for an initial solution. Each job J i is first assigned on a machine M k with the smallest p ik and allocated on a M k with the smallest r ki when p i1 = p i2 = · · · = p im , then scheduling string is randomly produced. The remaining N − 1 initial solutions are randomly generated.
After initial population P is produced, all solutions in P are sorted in the ascending order of C max , suppose that C x 1 max ≤ C x 2 max ≤ · · · C x N max ; then, x 1 is added into EB 1 , x 2 is included into EB 2 , x 3 is assigned into EB 1 , x 4 becomes a member of EB 2 and so on; finally, two employed bee swarms EB 1 , EB 2 are obtained, where C x i max indicates makespan of solution x i .

Employed Bee Phase with Adaptive Competition
Adaptive competition is performed between EB 1 and EB 2 based on their evolution quality, which is defined below.
where Evq gen EB j is the evolution quality of EB j on generation gen, if x i is replaced with z in the employed bee phase on generation gen, λ gen i is 1; otherwise 0. When EB 1 and EB 2 are compared, if Evq gen EB 1 > Evq gen EB 2 , then EB 1 obtains extra R searches from EB 2 , that is, EB 1 is given N/2 + R searches and EB 2 has N/2 − R searches. One search means that, for a solution x i , global search is first done and then a multiple neighborhood search of x i is executed.
For solution x i ∈ EB j , global search is shown as follows. Randomly choose a solution y ∈ EB j , execute two-point crossover between x i , y on machine assignment string, if the obtained solution z is better than x i , then update Θ with x i and replace x i with z; else perform two-point crossover between x i , y on scheduling string, if the produced solution z is better than x i , then update Θ with x i and replace x i with z.
Θ denotes a set of historical optimization data and is updated as follows. If |Θ| < |Θ| max , then solution z is directly included into Θ; otherwise, if z is better than the worst member of Θ, then the worst member is replaced with z, where |Θ| max indicates maximum size of Θ. We set |Θ| max to be 50 by experiments.
Neighborhood structures N 1 − N 5 are used. N 1 is depicted as follows. A randomly chosen job from a machine with the longest completion time is moved to a machine with smallest completion time. N 2 generates new solutions by deciding a randomly selected job on a machine M k with the longest completion time and a randomly chosen job on machine M l , l = k and swapping them. N 3 is shown below. Randomly decide two machines M k , M l , l = k and swap a randomly selected job J i on M k and a randomly chosen job J j on M l . In N 1 , N 2 , N 3 , only machine assignment string is changed. N 4 is applied to obtain new solutions by randomly deciding a machine M k and two jobs on M k and exchanging them. When N 5 is done, a M k , J i , J j on M k are randomly determined, ten θ i is inserted on the position j − 1 of scheduling string, if j = 1, θ i is inserted on position j. Multiple neighborhood search of x i is shown below. Let g = 1, repeat the following steps until g = 6: produce a solution z ∈ N g (x i ), if C z max < C x i max , then x i is replaced with z and g = 6; otherwise, g = g + 1.
com j is defined. Initial com j = 0, j = 1, 2. If EB j obtains extra seach times from EB 3−j , then com j = com j + 1 and com 3−j = 0. To avoid excessive competition, solution migration is executed if one of com 1 and com 2 exceeds or is equal to Q. If com j ≥ Q, then R solutions with smallest makespan are chosen from EB j and substitutes for the worst R solutions of EB 3−j , and reduced variable neighborhood search (RVNS) acts on each of R newly added solutions of EB 3−j , where Q is integer.
It can be found that when com j = Q, com 3−j must be 0, and thus only one of com 1 and com 2 exceeds or is equal to Q. RVNS is performed for solution x: let w = 1, g = 1, repeat the following steps until w > T: produce a new solution z ∈ N g (x), if z is better than x, then update Θ with y and replace y with z, and g = 1; otherwise, g = g + 1, let g = 1 if g = 6, where T is integer.
Employed bee phase is composed of two cases. If com 1 < Q and com 2 < Q, then the first case is executed; otherwise, the second case is performed. The first case is executed as follows.
(3) If Evq gen EB 1 > Evq gen EB 2 , then com 1 = com 1 + 1, com 2 = 0, sort all solutions of EB 1 and EB 2 , respectively, in the ascending order of makespan, let W be the set of R solutions with smallest makespan from EB 1 , execute one search for each solution in EB 1 and one search for each x ∈ W, sequentially, perform one search for each of the first N/2 − R solutions of EB 2 . (4) If Evq gen EB 2 > Evq gen EB 1 , then com 2 = com 2 + 1, com 1 = 0, EB 2 obtains extra R searches and EB 1 just has N/2 − R searches as done in (3). The second case is described below. If com j ≥ Q, R solutions with best makespan are chosen from EB j , for each chosen solution x ∈ EB j , if it is better than the worst solution of EB 3−j , then the worst solution y of EB 3−j is replaced with x and RVNS acts on y.
When the second case is executed, solution migration is applied, RVNS only acts on the transferred solutions from EB j with com j ≥ Q and searches of the first case are not done, as a result, evolution quality of EB 3−J can be improved and EB 3−j can win in the next competition with EB j . EB 1 and EB 2 compete according to evolution quality, excessive competition is considered and the worse employed bee swarm is improved, as a result, EB 1 and EB 2 can compete extensively.

Adaptive Onlooker Bee Phase and New Scout Phase
Adaptive onlooker bee phase is shown as follows.
For each onlooker bee l = 1, 2, · · · , N, select a x from the chosen empoyed bee swarm by roulette selection in Section 3, execute one search for the solution x.
where rand follows uniform distribution on [0, 1], f it(x i ) is equal to 1/C x i max . In the onlooker bee phase, a employed bee swarm is selected adaptively. IfC 1 max < C 2 max , then the probability of EB 1 is less than that of EB 2 and EB 2 has higher possibility than EB 1 in step (2), that is, EB 2 with lower solution quality is given higher selection possiblity, as a result, EB 2 may possess more searches, its solutions can be improved, and EB 2 can win in the next competition.
A new scout phase is described below.
(1) Sort all solutions of P in the ascending order of C then for each solution y ∈ Θ, compute its probability pr y ; then, select a solution x ∈ Ω by roulette selection based on pr; produce y g ∈ N g (x), g = 1, 2, 3, 4, 5 sequentially, the best y g directly substitutes for x i .
Otherwise, a set Φ with solutions x 1 , x 2 , · · · , x γ×N is constructed, and a solution x ∈ Phi is selected using the same way in the first case; we generate y 1 , y 2 , y 3 , y 4 , y 5 , and the best them becomes new x i .

Algorithm Description
The detailed steps of ABC-AC are shown as follows.
(1) Randomly produce initial population P with N solutions and divide the whole population into EB 1 and EB 2 , gen = 1. (2) Execute employed bee phase with adaptive competition.
A new scout phase is implemented based on the solution quality. Competitive and adaptive onlooker bee phase that lead to EB j have more chance to improve performance when com j ≥ Q andC j max >C 3−j max , as a result, EB j can win extra R search in the next employed bee phase, EB 1 and EB 2 can compete well and the possibility of falling local optimal can reduce notably; therefore, the search efficiency can be improved.

Computational Experiments
Many experiments are conducted to test the performance of ABC-AC for UPMSPR with PM. All experiments were implemented using Microsoft Visual C++ 2019 and run on 8.0G RAM 2.30 GHz CPU PC.

Test Instances and Comparative Algorithms
A total of 300 instances were used [4], which can be obtained directly from http: //soa.iti.es (accessed on 19 May 2022). R max = 5m, w k is an integer selected from the same interval as p ki , u k = round(w k + 3.5 × max i=1,2,··· ,n {p ki }). round(x) denotes an integer being closet to x. Five ways were used to produce processing time, and two ways were applied to generate additional resource; thus, 10 combinations of processing time and additional resources were used. No is defined as a combination of the a-th way of processing time and the b-th way of additional resource. b = 1 for No ≤ 5 and b = 2 for No > 5, and thus the instance is depicted as n × m × No.
TAFOA [3] and a multi-pass heuristic (MPH) [7] are chosen as comparative algorithm because they can be directly used to solve UPMSPR with PM. Salehi Mir and Rezaeian [47] presented a hybrid particle swarm optimization and genetic algorithm (HPSOGA), which can be directly applied to our UPMSPR after the decoding procedure of ABC-AC is adopted; therefore, we selected it as a comparative algorithm.
ABC is constructed to show the effect of new strategies of ABC-AC, which are adaptive competition, adaptive onlooker bee phase and new scout phase. In ABC, only one employed bee swarm and no adaptive competition is used in the employed bee phase, each onlooker bee selects a food source according to probability prob i and a randomly chosen neighborhood structure is performed on the selected food source. The scout phase of Section 3 is directly adopted.

Parameter Settings
ABC-AC has the following parameters: N, R, T, Q, γ, Limit and stopping condition. We found that ABC-AC can converge well with 0.3n seconds of CPU time, and 0.3n seconds CPU time also can be used as a stopping condition of comparative algorithms. Thus, 0.3n seconds of CPU time is used as the stopping condition.
Then, we apply the Taguchi method [48] to decide the settings for other parameters. We select instance 50 × 10 × 1. Table 1 gives the levels of each parameter. The orthogonal array L 27 (3 6 ) is tested. ABC-AC with each combination run 10 times independently for the chosen instance.  Figure 2 shows the results of MI N and S/N ratio, in which the S/N ratio is defined as −10 × log 10 (MIN 2 ) and MI N is the best solution found in 10 runs. As shown in Figure 2, it can be found that ABC-AC with following combination N = 100, Q = 4, R = 10, T = 10, γ = 0.3 and Limit = 10 can obtain better results than ABC-AC with other combinations; thus, the above combination is adopted. ABC has following parameters: N = 100 and Limit = 8. Parameter settings of three comparative algorithms are directly selected from references [3,7,47] except that the stopping condition. We also test these settings for each comparative algorithm by Taguchi method. The experimental results show that those settings of each comparative algorithm are still effective and comparative algorithms with those settings can produce better results than MPH, HPSOGA and TAFOA with other settings.

Results and Discussion
ABC-AC is compared with ABC and three comparative algorithms. Each of five algorithms randomly runs 10 times on each instance. The corresponding results of all algorithms are shown in Tables 2-7 and A1-A3, where AVG is the average value of 10 elite solutions obtained in 10 runs, and MAX is the worst of 10 elite solutions in 10 runs. Tables A1-A3 are listed in Appendix A. Figure 3 presents a mean plot with a 95% confidence interval. As shown in Tables 2-4, ABC-AC produces better MI N than or identical MI N with ABC on all instances; moreover, MI N of ABC is worse than that of ABC-AC by at least 20 on more than 210 instances. ABC-AC converges better than ABC. This conclusion also can be obtained from Figure 3. It also can be found from Tables 5-7 and A1-A3 and Figure 3 that ABC-AC performs significantly than ABC on AVG and MAX. ABC-AC produces smaller AVG and MAX than ABC on all instances and AVG and MAX of ABC-AC are notably less than those of ABC on all instances with n ≥ 30. ABC-AC significantly outperforms ABC on convergence, average result and stability; thus, it can be concluded that the usage of new strategies, such as adaptive competition has a positive impact on the performance of ABC-AC.        It can be found from Tables 2-4 that ABC-AC converges better than its comparative algorithms. ABC-AC produces smaller than or identical MI N with three comparative algorithms on 258 of 300 instances; moreover, MI N of ABC-AC is less than that of TAFOA by at least 20 on 241 instances, smaller than that of MPH by at least 20 on 66 instances and better than that of HPSOGA by at least 20 on more than 250 instances. ABC-AC has better convergence than MPH, HSPOGA and TAFOA. This conclusion can also be drawn from Figure 3.
As shown in Tables 5-7, ABC-AC obtains smaller AVG than or the same AVG as MPH, TAFOA and HPSOGA on 278 instances; moreover, AVG of ABC-AC is better than that of its all comparative algorithms by at least 10 on more than 160 instances. ABC-AC possesses better average performance than its three comparative algorithms. Figure 3 also depicts the average performance differences between ABC-AC and each comparative algorithm.
It also can be seen from Tables A1-A3 that MAX of ABC-AC exceeds that of three comparative algorithms on only 18 instances. ABC-AC has smaller MAX than comparative algorithms by at least 10 on 253 instances. Figure 3 also shows that ABC-AC possesses better stability than the comparative algorithms.
The good performance of ABC-AC results from its adaptive competition, adaptive onlooker bee phase and new scout phase. Adaptive competition and adaptive onlooker bee phase can effectively lead to the extensive competition between two employed bee swarms, which can be evolved fully. A new scout phase can effectively extend the exploitation ability of ABC-AC. These features can result in a low possibility of a falling local optimum and a good balance between exploration and exploitation; thus, ABC-AC is a competitive algorithm for UPMSPR with PM.

Conclusions and Future Research
The consideration of additional resources and PM leads to a high application value of the results of UPMSPR with PM, and competition between swarms is an effective path to intensify the performance of ABC. In this study, new adaptive competition was implemented, and ABC-AC was proposed to solve UPMSPR with PM. Two employed bee swarms were evaluated and compete with each other in an adaptive way to dynamically select one from two cases employed in the bee phase.
The first is the shifting of search resources from the employed bee swarm with lower evolution quality to another one, and the second is the migration of solutions from the employed bee swarm with higher evolution quality to another one. An adaptive onlooker bee phase was implemented, and a new scout phase was given. A number of experiments were conducted on 300 instances. The computational results demonstrate that the new strategies of ABC-AC are effective, and ABC-AC has promising advantages in solving the considered UPMSPR.
Additional resources (learning effect, deteriorating jobs, etc.) often exist in an unrelated parallel machine manufacturing process; UPMSPR with these constraints is a future research topic. We will attempt to solve the above problems using meta-heuristics, such as the imperialist competitive algorithm. Competition or cooperation among populations are effective paths to improve the performance of meta-heuristics with multiple populations, and we will apply new methods to implement competition or cooperation to obtain new meta-heuristics with a high search efficiency. The hybrid flow shop scheduling problem with additional resources is also a future topic of ours.