A Novel Shufﬂed Frog-Leaping Algorithm for Unrelated Parallel Machine Scheduling with Deteriorating Maintenance and Setup Time

: Unrelated parallel machine scheduling problems (UPMSP) with various processing constraints have been considered fully; however, a UPMSP with deteriorating preventive maintenance (PM) and sequence-dependent setup time (SDST) is seldom considered. In this study, a new differentiated shufﬂed frog-leaping algorithm (DSFLA) is presented to solve the problem with makespan minimization. The whole search procedure consists of two phases. In the second phase, quality evaluation is done on each memeplex, then the differentiated search processes are implemented between good memeplexes and other ones, and a new population shufﬂing is proposed. We conducted a number of experiments. The computational results show that the main strategies of DSFLA were effective and reasonable and DSFLA was very competitive at solving UPMSP with deteriorating PM and SDST.


Introduction
The parallel machine scheduling problem (PMSP) is a typical scheduling problem that can be categorized into three types: identical PMSP, uniform PMSP, and unrelated PMSP (UPMSP). As the generalization of the other two types, UPMSP has attracted great attention, and a number of results have been obtained to solve UPMSP with various processing constraints, such as random breakdown and random rework [1][2][3][4].
Preventive maintenance (PM) often exists in many actual manufacturing cases, can effectively prevent potential failures and serious accidents in parallel machines, and is often required to be considered in UPMSP. Regarding UPMSP with maintenance, Yang et al. [5] studied UPMSP with aging effects and PM to minimize the total machine load and proved that the problem remained polynomially solvable when a maintenance frequency on every machine is given.
Tavana et al. [4] presented a three-stage maintenance scheduling model for UPMSP with aging effects and multi-maintenance activities. Wang and Liu [6] proposed an improved non-dominated sorting genetic algorithm-II for multi-objective UPMSP with multiresources PM. Gara-Ali et al. [7] provided several performance criteria and different maintenance systems and gave a new method to solve the problem with deteriorating and maintenance. Lei and Liu [8] proposed an artificial bee colony (ABC) with division for distributed UPMSP with PM.
Deteriorating maintenance means that the length of maintenance activity is not constant and depends on the running time of the machine. UPMSP with deteriorating maintenance has also been studied. Cheng et al. [9] and Hsu et al. [10] provided some polynomial solutions. Lu et al. [11] considered UPMSP with parallel-batching processing, deteriorating jobs, and deteriorating maintenance and presented a mixed integer programming model and a hybrid ABC with tabu search (TS).
In many real-life industries, such as the chemical, printing, metal processing, and semiconductor industries, SDST often cannot be ignored [12]. UPMSP with SDST has been extensively addressed since the pioneering work of Parker et al. [13]. Kurz and Askin [14] proposed several heuristics. Arnaout et al. [15] designed an improved ant colony optimization with a pheromone re-initialization method. Vallada and Ruiz [16] presented a genetic algorithm to minimize the makespan. Lin and Ying [17] developed a hybrid ABC for UPMSP with machine-dependent setup times and SDST.
Caniyilmaz et al. [18] applied an ABC algorithm to solve UPMSP with processing set restrictions, an SDST, and a due date. Diana et al. [19] presented an improved immune algorithm by introducing a local search and a new selection operator. Wang and Zheng [20] proposed an estimation of distribution algorithm and gave five local search strategies. Ezugwu and Akutsah [21] proposed an improved firefly algorithm refined with a local search. Fanjul-Peyro et al. [22] presented an exact algorithm. Bektur and Sarac [23] introduced a TS and a simulated annealing algorithm for UPMSP with SDST, machine eligibility restrictions and a common server. Cota et al. [24] developed a multi-objective smart pool search algorithm for green UPMSP with SDST.
For UPMSP with PM and SDST, Avalos-Rosales et al. [25] developed an efficient metaheuristic based on a multi-start strategy to minimize the makespan, and Wang and Pan [26] presented a novel imperialist competitive algorithm with an estimation of distribution algorithm to optimize the makespan and total tardiness.
SDST and deteriorating maintenance are common processing constraints and often exist simultaneously in the real-life production process; however, the previous works mainly deal with UPMSP with one of these two constraints, few papers focus on UPMSP with maintenance and SDST [25,26] and UPMSP with deteriorating PM and SDST is hardly studied. It is necessary to investigate UPMSP with deteriorating PM and SDST due to their extensive existences in production. On the other hand, meta-heuristics, including ABC, have been applied to solve UPMSP with various processing constraints, such as PM and SDST. As a meta-heuristic, by observing, imitating, and modeling the search behavior of frogs for the location with the maximum amount of available food, the shuffled frog-leaping algorithm (SFLA) is seldom used to handle UPMSP.
SFLA has a fast convergence speed and effective algorithm structure containing local search and global information exchanges [27]. It has been widely applied to solve various optimization problems, such as topology optimization and production scheduling problems [28][29][30][31][32][33][34][35][36][37][38][39][40][41][42][43][44][45]. The existing works on scheduling problems revealed that SFLA has great potential in solving UPMSP with deteriorating PM and SDST. On the other hand, the same search process and parameters are adopted in all memeplexes, and the differentiated search process is seldom used, which can effectively intensify the exploration ability and avoid falling local optima; thus, it is necessary to investigate the possible applications of SFLA with new optimization mechanisms for UPMSP with SDST and PM.
In this study, UPMSP with deteriorating PM and SDST is considered, and a new differentiated shuffled frog-leaping algorithm (DSFLA) is applied to minimize the makespan. The entire search procedure is composed of two phases. In the second phase, the memeplex quality is evaluated on each memeplex to divide all memeplexes into good memeplexes and others, then the differentiated search processes are implemented between good memeplexes and others, and a new population shuffling is proposed. We conduct experiments to test the effect of the main strategies and the search advantages for DSFLA.
The remainder of the paper is organized as follows. The problem is described in Section 2 followed by an introduction to SFLA in Section 3. DSFLA for the considered problem is reported in Section 4. Numerical experiments on DSFLA are reported in Section 5, the conclusions are summarized in the final section, and some topics of future research are provided.

Problem Description
UPMSP with deteriorating PM and SDST is composed of n jobs J 1 , J 2 , · · · , J n and m unrelated parallel machines M 1 , M 2 , · · · , M m . Each job can be processed on any one of m machines. The processing time p kj of job J j depends on the performance of its processing machine M k . The processing times on different machines are usually different.
On machine M k , job is processed in a time interval between two consecutive maintenance activities, and the length of the interval is indicated as u k , and w k denotes the duration of each maintenance. For deteriorating maintenance, w k is not constant and depends on M k and the starting time of maintenance, w k = c k + d k × t k , where c k , d k are constant, and t k indicates the starting time of maintenance on M k . There are some intervals for processing on each machine. If the processing of a job cannot be completed in a processing interval, the job cannot be processed in the current interval and should be moved to the next interval.
For SDST, s kij is the setup time for processing job J j after job J i on machine M k , s k0j indicates the setup time of machine M k to process the first job J j after a maintenance activity, and s kj0 is the setup time of machine M k to perform a maintenance activity after the job J j .
There have the following constraints on jobs and machines.
Each job and machine is available at time zero. Each job can be processed on only one machine at a time.
Operations cannot be interrupted.
Preemption is not allowed.
The problem is composed of the scheduling sub-problem and machine assignment sub-problem. The goal of the problem is to minimize the makespan.
Let C π(j) be a completion time of job j in schedule π, and the makespan can be defined by C max (π) = max j=1,...,n {C π(j) }. Thus, the objective is to find a schedule π that minimizes the makespan C * max = min π∈Π {C max (π)}, where Π is the set of all feasible schedules π. An illustrative example is provided. Tables 1 and 2 give the processing time and setup time. There are two machines and eight jobs. Data on deteriorating PM are shown in Figure 1, where c k = 1 and d k = 0.1 for all machines.  When no PM is considered, any two jobs on any one machine are symmetrical, that is, exchanging them does not change the makespan. When PM is handled, any two jobs on a machine between time 0 and the first PM or two consecutive PMs, because of the above, are reasonable; thus, the consideration of PM has impact on the optimization of the considered UPMSP.

Introduction to SFLA
In SFLA, a solution is defined as the position of a frog, and there is a population of possible solutions defined by a set of virtual frogs. After the initial population P is produced, the following steps, which are population division, memeplex search, and population shuffling, are repeated until the stopping condition is met.
Population division is as follows. After all solutions are sorted, suppose that Fit 1 ≥ Fit 2 ≥ · · · ≥ Fit N , and then solution x k is allocated into memeplex k(mod s) + 1, where k(mod s) indicates the remainder of k s, Fit i is the fitness of solution x i , and s indicates the number of memeplexes.
The search process in memeplex M l is shown below. x w is used as optimization object, then a new solution x w is produced by Equation (2) with x w and x b . If the new one is better than x w , then replace x w with x w ; otherwise, x w and x g are used to generate a solution x w by Equation (3). If x w has better fitness than x w , then x w becomes the new x w ; otherwise, a randomly obtained solution directly substitutes for x w , where x w , x b and x g are the worst solution and best solution in memeplex M l and the best solution of P.
where rand is a random number following uniform distribution in [0.1].
A new population P is constructed by shuffling all evolved memeplexes. As stated above, all memeplexes are often evolved by the same search process and parameters [29,32,33] and the differentiated search in memeplexes is seldom considered. When the differentiated search operators and parameters are introduced, the search ability will be intensified, and local optima can be effectively avoided, as a result, the search efficiency is greatly improved. In this study, DSFLA is presented to solve UPMSP with deteriorating PM and SDST.

DSFLA for UPMSP with Deteriorating PM and SDST
DSFLA is composed of two phases, and the differentiated search is implemented in the second phase.

Initialization, Population Division, and the First Phase
UPMSP consists of two sub-problems: machine assignment and scheduling, and twostring representation is often applied to indicate the solution of UPMSP [46,47]; however, two strings are often dependent each other, and it is difficult to design and apply global search or local search on each string independently. In this study, a solution of the problem is represented as a machine assignment string M θ 1 , M θ 2 , · · · , M θ n and a scheduling string [q 1 , q 2 , · · · , q n ], where M θ j is the assigned parallel machine for job J j , j = 1, 2, · · · , n, and q l is real number and corresponds to J l . These two strings are independent.
Lei and Liu [8] analyzed why a scheduling string is introduced because of the above mentioned changes of symmetry. The decoding process is described below. First, we decide on a machine for each job according to each machine assignment string, and then on each machine M k , for all jobs J i , J i+1 , · · · , J j allocated on M k -that is, The processing sequence of these jobs is decided by the ascending order of q l , l ∈ [i, j], i < j, and they process jobs and deal with maintenance on M k sequentially.
After the initial population P is randomly produced, population division is performed in the following way. Decide the best s solutions from P and sort them in the descending order of their objective. Then, the first solution is allocated into memeplex M 1 , the second solution is assigned into M 2 , and so on. Then, binary tournament selection is used to allocate other solutions into memeplexes: randomly select two solutions x i and x j , and then, if If two solutions have the same objective, then stochastically choose one of them and add it into M 1 . The unchosen solution goes back to population P. Repeat the above steps for deciding a solution for M 2 , M 3 , · · · , M s , and then repeat the above procedure until all solutions are assigned. Obviously, N = s × θ, where θ denotes the size of each memeplex.
There are two phases in the search process of DSFLA. The steps of the first phase are identical with SFLA in Section 3. The search process in M i is shown below. Repeat the following steps R 1 times: decide x w , x b ∈ M i , execute two-point crossover on machine assignment string of x w and x b , if the obtained solution x is better than x w , then replace x w with x; otherwise, apply two-point crossover on a scheduling string between x w and x b . If the generated solution x has a smaller makespan than x w , x becomes the new x w , where R 1 is an integer.
In the first phase, global search is only used because of its good exploration ability in the early search stage. In the second phase, the differentiated search processes are implemented based on memeplex quality evaluation.

The Second Phase
The evaluation of memeplex quality is seldom considered in SFLA. In this study, memeplex quality is evaluated according to solution quality and evolution quality. For memeplex M l , its quality Meq l is defined by where α 1 , α 2 are real number, msq l and mvq l indicate solution quality and evolution quality of M l , respectively, msq max = max Solutions of H 1 are better than those of H 2 ; therefore, we set β 1 > β 2 to reflect this feature. β 1 = 0.4 and β 2 = 0.1 are obtained by experiments.
Let Im x indicate the improved number of x between the first generation and the current generation. When x ∈ M l is chosen as an optimization object, such as x w , in general SFLA, if an obtained solution x is better than x, then Im x = Im x + 1. Se x is the total search times from the first generation to the current generation.
For solution x i , its act x i is used to evaluate its evolution quality and is computed by The second phase is shown as follows.
(1) Perform population division, compute Meq l for all memeplexes, sort them in descending order of Meq l , and construct set Θ = M l Meq l > Meq, l ≤ η × θ . (2) For each memeplex M l , M l / ∈ Θ, repeat the following steps R 1 times if |T | > 0, execute global search between x b and a randomly chosen y ∈ T ; else perform global search between x b and a solution y ∈ M l with act y ≥ act x for all x ∈ M l .
(3) For each memeplex M l ∈ Θ, 1 sort all solutions in M l in the ascending order of makespan, suppose C max ( Repeat the following steps R 2 times, and randomly choose a solution x i ∈ M l \ϕ; if act x i > 0.5, then select a solution y ∈ ϕ by roulette selection based on Pr y , execute global search between x i and y, and update memory T ; else execute global search between x i and a solution z with act z ≥ act x i for all x i ∈ M l and update memory T . where η is a real number and set to be 0.4 by experiments, Meq indicates the average quality of all memeplexes, Θ is the set of good memeplexes, and Pr y is a probability and defined by where rank y is an integer and decided by ranking according to makespan in the first step of step (3) in the above Algorithm.
In the second phase, after all memeplexes are sorted in the descending order of Meq l , suppose Meq 1 ≥ Meq 2 ≥ · · · Meq s . Memory T is used to store the intermediate solutions. The maximum size |T | max is given in advance. We set |T | max to be 200 by experiments. When the number of solutions exceeds |T | max , a solution x can be added into T when x is better than the worst solution of T and substitutes for the worst one.
Six neighborhood structures are used. N 1 is shown below. Randomly select a job from the machine M k with the largest C k max and move it into the machine M g with the smallest C g max , where C k max and C g max are the completion time of the last processed job on M k and M g , respectively. N 2 is performed in the following way. Decide on a machine M k with the largest C k max and a job J i with the largest processing time p ki on M k , randomly choose a machine M g , g = k and a job J j with the largest p gj and exchange J i and J j between M k and M g . N 3 is described as follows. Randomly select two machines M k and M g and exchange a job J i with the largest p ki and a job J j with the largest p gj between these two machines. N 1 , N 2 , N 3 only act on the machine assignment string. N 4 ,N 5 ,N 6 are performed on a scheduling string by swapping two randomly chosen genes, inserting a randomly selected gene into a new randomly decided position, and inverting genes between two stochastically positions k 1 , k 2 , k 1 < k 2 .
Multiple neighborhood search is performed in the following way. For solution x, let u = 1, repeat the following steps V times: produce a solution z ∈ N u (x), u = u + 1, let u = 1 if u = 7, and if z is better than x, then replace x with z and Im x = Im x + 1.
Global search is executed in the same way of the first phase.
In the existing SFLA [29,32,33], a new population P is constructed by using s evolved memeplexes. In this study, new population shuffling is done in the following way: γ best solutions of T and s memeplexes are added into new population P, and γ worst solutions of P are removed. We test by experiments and set γ = 0.1 × |T | max . Some worst solutions of P can be updated by solutions of T , that is, solutions of P can be improved by memeplex search or shuffling.
In the second phase, the set Θ is composed of good memeplexes with better quality than other memeplexes, in the search process for a good memeplex, a global search of optimization object x is implemented according to act x , and then multiple neighborhood search acts on the solutions in ϕ. Only global search is executed for other memeplexes; moreover, different parameters, R 1 , R 2 , R 1 = R 2 , are used, and, as a result, differentiated search is implemented.

Algorithm Description
The detailed steps of DSFLA are shown below.
(1) Initialization. Randomly produce initial population P with N solutions, and let initial T be empty. Unlike the previous SFLA [29,32,33], DSFLA has the following features. (1) Quality evaluation is done for all memeplexes according to the solution quality and evolution quality and all memeplexes are categorized into two types: good memeplexes and other memeplexes. (2) The differentiated search is implemented by different search strategies and parameters for two types of memeplexes; as a result, the exploration ability is intensified; and the possibility of falling into local optima diminishes greatly.

Computational Experiments
Extensive experiments were conducted on a set of instances to test the performance of DSFLA for UPMSP with deteriorating PM and SDST. All experiments were implemented by using Microsoft Visual C++ 2019 and run on 8.0 G RAM 2.30 GHz CPU PC.
The general SFLA in Section 3 was also implemented, in which global search between two solutions is performed in the same way as the first stage of DSFLA. The comparison between DSFLA and SFLA is to show the effect of the main strategies of DSFLA.

Parameter Settings
In DSFLA, the stopping condition is the maximum number max_it of objective function evaluations. We found that DSFLA can converge fully. We also tested this condition for other comparative algorithms when max_it is 10 5 . We also found that the above max_it was appropriate; thus, we used this stopping condition.
DSFLA possesses other main parameters: N, s, R 1 , R 2 , V, and max_it 1 , where max_it 1 denotes the maximum number of objective function evaluations in the first phase. The Taguchi method [49] was used to decide the settings for parameters. We selected instance 150 × 20 for parameter tuning. The levels of each parameter are shown in Table 3. There were 27 parameter combinations according to the orthogonal array L 27 (3 6 ). DSFLA with each combination run 10 times independently for the chosen instance. The results of MI N and the S/N ratio are shown in Figure 2, in which the S/N ratio is defined as −10 × log 10 (MIN 2 ) and MI N is the best solution found in 10 runs. From Figure 2, DSFLA with following combination N = 80, s = 5, R 1 = 50, R 2 = 100, V = 240 and max_it 1 = 10 4 obtained better results than DSFLA with other combinations; therefore, the above combination was adopted.

Results and Analyses
DSFLA was compared with SFLA, ABC-TS, HPSOGA, and MSA. All parameters except the stopping conditions of ABC-TS, HPSOGA, and MSA were directly adopted from the original references. For SFLA, there were no R 1 , R 2 , V and max_it 1 , and the other parameters were given the same settings as DSFLA. Each algorithm randomly ran 10 times for each instance. Tables 4-6 show the computational results of all algorithms, in which AVG is the average value of the obtained 10 elite solutions in 10 runs, and SD is the standard deviation of 10 elite solutions.     Table 7 describes the computational times of DSFLA and its comparative algorithms, in which the unit of time is seconds. Figure 3 gives a box plot of all algorithms, in which DM (DA)(DS) indicates the MI N (AVG) (SD) of an algorithm minus the MI N (AVG) (SD) of DSFLA. Figure 4 reveals the convergence curves of two instances.   As shown in Tables 4-6, SFLA could not produce better MI N than DSFLA on any instances and obtains bigger MI N than SFLA by at least 100 for most instances. DSFLA had better convergent performance than SFLA. DSFLA generated smaller AVG than SFLA on all instances, and the differences of AVG between the two algorithms were significant. DSFLA performed better than SFLA for the average performance. The SD of SFLA was also worse than that of DSFLA for most instances, and SFLA was inferior to DSFLA regarding search stability. DSFLA performed notably better when compared with SFLA. This conclusion can also be drawn from Figure 3; thus, new strategies, such as differentiated search, had a positive impact on the performance of DSFLA.
It can be seen from Table 4 that DSFLA produced smaller MI N compared with HPSOGA and ABC-TS for nearly all instances and generated a worse MI N than MSA for only 11 instances. DSFLA converged better than its comparative algorithms. The convergence performance differences between DSFLA and its comparative algorithm can also be seen from the box plot and convergence curves in Figures 3 and 4.
The results in Table 5 show that DSFLA obtained a better AVG over HPSOGA and ABC-TS for nearly all instances and possessed a smaller AVG than MSA for most instances. DSFLA had a better average performance than its three comparative algorithms. This conclusion can also be drawn from Figure 3. Table 6 and Figure 3 reveal that DSFLA had better stability than its three comparative algorithms; thus, we concluded that DSFLA can provide promising results for solving UPMSP with deteriorating PM and SDST.
The good performances of DSFLA mainly resulted from its new strategies in the second phase. The differentiated search was implemented by memeplex quality evaluation and different search combinations of global search and multiple neighborhood search. These strategies effectively intensified the exploration ability and avoided the search falling into local optima. As a result, a high search efficiency was obtained; thus, DSFLA is a very competitive method for the considered UPMSP.

Conclusions
UPMSP with various processing constraints has been extensively considered. This paper addressed UPMSP with deteriorating PM and SDST and provided a new algorithm named DSFLA to minimize the makespan. In DSFLA, two search phases exist, memeplex quality evaluation is performed, and the differentiated search processes between two kind of memeplexes are implemented in the second phase. A new population shuffling was also presented. A number of computational experiments were conducted. The computational results demonstrated that DSFLA had promising advantages, such as good convergence and stability in solving the considered UPMSP. UPMSP with at least two actual constraints, such as PM, SDST, and learning effects, may attract attention. We will continue to focus on these UPMSP by using meta-heuristics, including ABC and the imperialist competitive algorithm. Uncertainty often cannot be neglected and should be added into scheduling problems. UPMSP with uncertainty and energy-related elements, etc. is our future topic. Reinforcement learning has been used to solve scheduling problems and we will pay attention to meta-heuristics with reinforcement learning for UPMSP with various processing constraints.