An Investigation on a Closed-Loop Supply Chain of Product Recycling Using a Multi-Agent and Priority Based Genetic Algorithm Approach

: Product recycling issues have gained increasing attention in many industries in the last decade due to a variety of reasons driven by environmental, governmental and economic factors. Closed-loop supply chain (CLSC) models integrate the forward and reverse ﬂow of products. Since the optimization of these CLSC models is known to be NP-Hard, competition on optimization quality in terms of solution quality and computational time becomes one of the main focuses in the literature in this area. A typical six-level closed-loop supply chain network is examined in this paper, which has great complexity due to the high level of echelons. The proposed solution uses a multi-agent and priority based approach which is embedded within a two-stage Genetic Algorithm (GA), decomposing the problem into (i) product ﬂow, (ii) demand allocation and (iii) pricing bidding process. To test and demonstrate the optimization quality of the proposed algorithm, numerical experiments have been carried out based on the well-known benchmarking network. The results prove the reliability and e ﬃ ciency of the proposed approach compared to LINGO and the benchmarking algorithm discussed in the literature.


Introduction
The closed-loop supply chain (CLSC) has become more popular in recent years due to several reasons.One of the most prominent reasons is that environmental issues have gained increasing attention.European regulations have increased a producer's responsibility in several branches of industry, such as WEEE (waste electrical and electronic equipment) 2001 for consumer electronics [1].Customers also expect to trade-in old products when buying new ones, making producers pay more attention to the reversion of used products or materials.Another crucial reason for operating in the CLSC is the cost.A well-designed CLSC network can provide significant cost savings in procurement, inventory, transportation and recovery processing.Hewlett-Packard estimated that the cost of returns was as much as 2% of their gross outbound sales [2].
Guide and Van Wassenhove [3] defined closed-loop supply chain management (CLSCM) as the design, control and operation of a system to creat maximum value in the entire life cycle of a product, and dynamically restore value from different types and volumes of returns over time.The management of CLSC describes the discipline of optimizing the delivery of goods, services and information from supplier to customer and simultaneously from customer recovery to supplier [4].With the same principle as in supply chain management, the goals of closed-loop supply chain management typically include transportation network design, location of manufacture, distribution or dismantling center, product recycling and other efforts to improve cost savings.Location selection and transportation network design are two of the most important fields of CLSCM.
Many research studies related to the recovery of products have been conducted in the filed of the supply chain including both academic and industrial applications.Fleischmann et al. [5] and Wells and Seitz [6] believed that the forward and reverse logistics chains should be combined in operations and the flow of materials to establish a closed-loop supply chain.The practical applications of CLSC can be found in many fields.Krikke, Bloemhof-Ruwaard and Van Wassenhove [1] developed a quantitative model which was applied to a refrigerators' CLSC network design of a Japanese company concerning its European strategies.Abu Bakar and Rahimifard [7] investigated into a real case of the WEEE recycling process and indicated that systematic planning for an individual product's recycling process will improve the value of recovery significantly.Olugu and Wong [8] implemented a CLSC performance evaluation system for an automotive industry company and reduced the cost of the whole CLSC network significantly.Yan and Yan [9] investigated the open loop and closed loop supply chain for product remanufacturing and proved the effectivity of closed loop reverse logistics through numerical examples.
In the literature, it can be seen that the research of CLSC covers many aspects.In this paper, the focus is the optimization of the CLSC network.The CLSC network links the forward logistics and reverse logistics, in which the reverse part is connected by the action of customer recovery.It focuses on the design of transportation routes and decisions on the facility operational state, instead of investigation into the interactions between demands and returns or uncertainties [10,11].The focused CLSC network problem covers both the transportation and facility location problems at several stages.Firstly, the transportation problem (TP) was originally proposed and solved by Hitchcock [12] and became a well-known basic network problem.The facility location problem has received much attention since 1985, with the aim of deciding on the number of distribution centers and finding good locations so as to satisfy customer demand at minimum facility operation costs and delivery costs [13].Since the CLSP problem contains the capacitated p-median facility location problem, it is also NP-hard [14].
In this academic area, most of these NP-hard CLSC problems are formulated into linear or nonlinear problems [15].To solve these kinds of NP-hard problems, exact algorithms can be an option but for large scale problems, the computational time is too long and not useful practically.Hence, heuristic algorithms such as the Genetic Algorithm (GA) have become an efficient method and have gained more popularity.Although GA can solve this kind of NP-hard problem more efficiently than exact algorithms, especially for large scale ones, the genetic seeking ability has not been exploited sufficiently.In the literature, there are several kinds of adapted GA to solve the above mentioned problem but all of the encoding processes are single-stage.The encoding of these adapted GA is mainly classified into two categories.One with chromosomes only expresses the transportation route and facilities operation state, not considering the product flows.These methods separate the problem and use GAs to solve part of it.The others, with chromosomes, express all the information including transportation route, product flows and facilities operation state.Since one chromosome in a single stage contains vast information, the genetic operations will become difficult due to a huge number of combinations.It will also dramatically decrease the genetic searching ability and increase the calculation time.Chen, Chan and Chung [16] proposed a two-stage priority based GA and implemented a basic scale experiment to show the feasibility of overcoming the above mentioned problem.
In this investigation, a priority and multi-agent based two-stage encoding GA approach is developed, which can resolve the focused CLSC network optimized problem.This algorithm consists of two stages, decomposing the CLSP into two sub-problems.The novel two-stage priority based encoding enhances the genetic searching ability of GA when solving this type of location and transportation problem.In the numerical calculation experiments, the proposed approach is adopted to deal with problems at different scales and comparisons with single-stage encoding GA and LINGO are also implemented.In Experiment 1, the same dataset used by Wang and Hsu [17] is used for comparison.In Experiments 2 and 3, the data sets are generated randomly and available for readers.The results show that the proposed approach can be successfully used to solve the integrated closed-loop problem.

Literature Review
In this academic area, many researchers have conducted extensive study on the optimal CLSC network.Barros et al. [18] developed a heuristic algorithm to optimize a location model considering product recycling.Sheu, Chou and Hu [10] solved a comprehensive CLSC model with a multi-objective linear programming.Paksoy et al. [19] balanced the equilibrium between various costs using a linear programming model.Özkır and Başlıgıl [20] formulated an integer linear programming model to optimize a recovery process in CLSC.Amin and Zhang [21] introduced a multi-objective mixed-integer model to determine the uncertainty of the CLSC configuration and selection process simultaneously.Atabaki et al. [22] developed a firefly algorithm to solve large scale problems in CLSC network design with price sensitive demand.
Besides using linear programming for formulation, genetic algorithm has become popular as an efficient method to deal with problems with NP-hardness.Lee and Chan [23] optimized the location problem of collection points using genetic algorithm.Kannan et al. [24] studied a multi-level product return network model in CLSC with genetic algorithm.Zhang et al. [25] developed a genetic algorithm to solve a CLSC strategy problem with remanufacturing.Lee and Lee [26] proposed an optimization approach integrate fuzzy control with genetic algorithm to solve a CLSC model, which efficiency was demonstrated by a real data experiment.Akbari et al. [27] formulated a two-level supply chain network, and solved with three approaches as genetic algorithm and two meta-heuristics.
In the particular area of focused CLSC network optimization, GA is also widely used.Min et al. [28] formulated a nonlinear programming to describe the location problem for return points in CLSC, and solved it using a binary chromosome encoding genetic algorithm, which can only deal with the open and shut information of facilities.Gen, Altiparmak and Lin [4] proposed a two-stage transportation problem, and optimized it with a priority-based encoding genetic algorithm.Their encoding method conveyed information of product flow and transportation route, without considing flow allocation.Similarly, Wang and Hsu [17] designed a spanning-tree based genetic algorithm to optimize an integrated CLSC network.The product flow information was added by a spanning-tree method to the chromosome encoding.Tuzkaya et al. [29] proposed a genetic algorithm with chromosome encoding of binary strings, representing the collection periods and states of facilities, to optimize the white goods recovery network.Yun, Chuluunsukh and Gen [30] proposed a hybrid genetic algorithm approach to solve the sustainable closed-loop supply chain design problem and prove it outperforms its competitors.
From the literature, it can be seen that in solving the CLSC problem with GA, most of the encoding methods included either the delivery route information or the freight volume information only.While in the CLSC problem, the delivery route decision and freight volume decision are integrated with each other.Hence an integrated genetic algorithm which can solve the transportation route design problem and the product flow allocation problem simultaneously may improve the performance of GA in solving this kind of CLSC problem.In this research, a multi-agent and priority based GA is proposed to optimize the CLSC network compresensively.The numerical experiments provide different scales of problems to test the proposed approach.It also been compared with single-stage encoding GA and LINGO.The experiments results prove the high quality performance and efficiency of the proposed approach.

Problem Descriptions
In this research, the model established by Wang and Hsu [17] is used to describe the CLSC problem.Six levels are included in this CLSC as shown in Figure 1.In Figure 1, the three solid lines represent three levels of product flow in the forward chain and the three dotted lines represent the three levels of product recycle flow in the reverse chain.In the forward chain, customer demands are given and the distribution centers transport the finished products to customers to meet the demand.Manufacturers transport the corresponding finished products to the distribution centers.In order to produce sufficient products, manufacturers need enough raw materials.In this CLSC, both suppliers and dismantlers can provide raw materials to the manufactures.The raw materials provided by dismantlers are from the recycling of used products collected by the distribution centers from customers.There are many different manufacturers, distribution centers and dismantlers located in different places.Each transportation route has its own unit shipping cost.The purpose of this model is to optimize the transportation network where the facilities are located.The notation and functions are defined as follow.In Figure 1, the three solid lines represent three levels of product flow in the forward chain and the three dotted lines represent the three levels of product recycle flow in the reverse chain.In the forward chain, customer demands are given and the distribution centers transport the finished products to customers to meet the demand.Manufacturers transport the corresponding finished products to the distribution centers.In order to produce sufficient products, manufacturers need enough raw materials.In this CLSC, both suppliers and dismantlers can provide raw materials to the manufactures.The raw materials provided by dismantlers are from the recycling of used products collected by the distribution centers from customers.There are many different manufacturers, distribution centers and dismantlers located in different places.Each transportation route has its own unit shipping cost.The purpose of this model is to optimize the transportation network where the facilities are located.The notation and functions are defined as follow.Recovery quantity at DC k from customer l α j = 1 if production takes place at manufacturer j 0 otherwise

Indices
Subject to , ∀l : ceiling for Gauss' symbol (7) The objective function ( 1) is constructed to minimize the total cost including transportation and operatation costs.The capacity limitation of the suppliers and manufacturers is stipulated by the Equations ( 2) and (3).Equation ( 4) is used to ensure that the total capacity of DC can take account of the gross flows, forwards and backwards.Equations ( 5) and ( 6) formulate the capacity limitation of the distribution ceters and dismantlers in reverse logistics.Equation ( 7) is used to express the relationship between recovery rate and customer recovery.The equality of the in-flow and the out-flow is guaranteed by the Equations ( 8), ( 9), ( 10) and (11).Equation ( 12) is used to guarantee that customer needs are met.Equations ( 13) and ( 14) respectively show the binary and integer variables.
The CLSC is modeled by using the multi-agent based approach shown in Figure 2. It consists of 5 types of agents: Supplier Agents (Si), Manufacturing Agents (Mj), Distribution Center Agents (DCk), Customer Agents (Cl) and Dismantler Agent (dm).The operation of the system is driven by the demands coming from customers, which are supplied by the DCs.Meanwhile, DCs are supplied by the manufacturers, who will then determine the production quantity and further initiates demand to suppliers.In this CLSC multi-agent model, this demand information will only be shared between two layers, such as the agents in the layer of S with M, M with DC, DC with C, DC with d and d with M in order to simplify the problem complexity.Accordingly, we propose a novel two-stage priority based Genetic Algorithm (GA) for this multi-agent modeling approach.The algorithm deals with 2 main issues, (i) pairing up each pair of agents between every layer and (ii) determining the actual quantity.

 
, , , , , 0 , , , , The objective function ( 1) is constructed to minimize the total cost including transportation and operatation costs.The capacity limitation of the suppliers and manufacturers is stipulated by the Equation ( 2) and (3).Equation ( 4) is used to ensure that the total capacity of DC can take account of the gross flows, forwards and backwards.Equation ( 5) and ( 6) formulate the capacity limitation of the distribution ceters and dismantlers in reverse logistics.Equation ( 7) is used to express the relationship between recovery rate and customer recovery.The equality of the in-flow and the outflow is guaranteed by the Equation ( 8), ( 9), ( 10) and (11).Equation ( 12) is used to guarantee that customer needs are met.Equation ( 13) and ( 14) respectively show the binary and integer variables.
The CLSC is modeled by using the multi-agent based approach shown in Figure 2. It consists of 5 types of agents: Supplier Agents (Si), Manufacturing Agents (Mj), Distribution Center Agents (DCk), Customer Agents (Cl) and Dismantler Agent (dm).The operation of the system is driven by the demands coming from customers, which are supplied by the DCs.Meanwhile, DCs are supplied by the manufacturers, who will then determine the production quantity and further initiates demand to suppliers.In this CLSC multi-agent model, this demand information will only be shared between two layers, such as the agents in the layer of S with M, M with DC, DC with C, DC with d and d with M in order to simplify the problem complexity.Accordingly, we propose a novel two-stage priority based Genetic Algorithm (GA) for this multi-agent modeling approach.The algorithm deals with 2 main issues, (i) pairing up each pair of agents between every layer and (ii) determining the actual quantity.

A Two-Stage Priority-Based Genetic Algorithm Encoding with Multi-Agent
In this proposed CLSC problem, the objective is to identify an applicable delivery route and corresponding product flow volume to obtain the minimum cost of transportation and facility operation.Here two problems have to be solved simultaneously; one is to decide the handling quantity of each active facility and the other is to decide the transportation route among the whole network.Additionally, these two problems are under a six-level CLSC which makes the problem more complicated.
By using the single-stage encoding GA, these two problems have to be expressed in a single chromosome, which means the chromosome must be able to convey the information on the route and volume and also the active state of the facilities.These vast amounts of information will make the

A Two-Stage Priority-Based Genetic Algorithm Encoding with Multi-Agent
In this proposed CLSC problem, the objective is to identify an applicable delivery route and corresponding product flow volume to obtain the minimum cost of transportation and facility operation.Here two problems have to be solved simultaneously; one is to decide the handling quantity of each active facility and the other is to decide the transportation route among the whole network.Additionally, these two problems are under a six-level CLSC which makes the problem more complicated.
By using the single-stage encoding GA, these two problems have to be expressed in a single chromosome, which means the chromosome must be able to convey the information on the route and volume and also the active state of the facilities.These vast amounts of information will make the genetic operation too complicated to handle because of too many possible combinations.In addition, this chromosome structure will obviously increase the calculation time.Since all the information is considered at the same time, the search is easy to get lost in the local optimum.
To deal with the problem, a two-stage priority-based GA with multi-agent approach is established.The proposed approach divides the encoding process into two stages, which express the above mentioned two problems in the CLSC network respectively.Figure 3 shows the flow diagram of this developed approach.
From Figure 3, it can be observed that this GA approach has two stages in the encoding process including Stage 1 "Route Decision", Stage 2 "Freight Volume Decision".Route Decision stage is used to decide the delivery route in the proposed CLSC network.It also applied to determine the pairing of agents.Freight Volume Decision stage is applied to determine the freight volume for each selected route according to the results of stage 1.These two stages are carefully described in Sections 4.1 and 4.2 respectively.The "Cost Rank" process before "Route Decision" is the preparation of the priority-based "Freight Volume Decision" which is explained in Section 4.2.The right part of Figure 3 describes the genetic operations.Section 4.3 gives details.
To deal with the problem, a two-stage priority-based GA with multi-agent approach is established.The proposed approach divides the encoding process into two stages, which express the above mentioned two problems in the CLSC network respectively.Figure 3 shows the flow diagram of this developed approach.
From Figure 3, it can be observed that this GA approach has two stages in the encoding process including Stage 1 "Route Decision", Stage 2 "Freight Volume Decision".Route Decision stage is used to decide the delivery route in the proposed CLSC network.It also applied to determine the pairing of agents.Freight Volume Decision stage is applied to determine the freight volume for each selected route according to the results of stage 1.These two stages are carefully described in Section 4.1 and Section 4.2 respectively.The "Cost Rank" process before "Route Decision" is the preparation of the priority-based "Freight Volume Decision" which is explained in Section 4.2.The right part of Figure 3 describes the genetic operations.Section 4.3 gives details.

Stage 1-Route Decision
The first stage of encoding is to decide the applicable delivery route.In the first stage, the chromosome generated consist of binary genes, which only represent the information of delivery route.Since the chromosomes, in stage 1 only deliver the information of transportation route except for the quantity of product flow, it makes the genetic operations, especially the validation, much more concise than the traditional GA.
The chromosome possesses six sections corresponding to the six levels of the CLSC model.In each section, the amount of genes is equal to the product of the amount of suppliers and the amount of demanders.In general, the number of genes in a chromosome, is expressed by where I, J, K, L and M are the number of suppliers, manufacturers, distribution centers, customers and dismantlers respectively.
To explain the proposed algorithm clearly, a numerical example is provided.In this example, I = 3, J = 5, K = 2 which means that three suppliers provide materials to five manufacturers and these five manufacturers provide finished products to two DCs.The chromosome at this stage in the example is shown in Figure 4 and the corresponding delivery route is shown in Figure 5a.
route.Since the chromosomes, in stage 1 only deliver the information of transportation route except for the quantity of product flow, it makes the genetic operations, especially the validation, much more concise than the traditional GA.
The chromosome possesses six sections corresponding to the six levels of the CLSC model.In each section, the amount of genes is equal to the product of the amount of suppliers and the amount of demanders.In general, the number of genes in a chromosome, is expressed by , where I, J, K, L and M are the number of suppliers, manufacturers, distribution centers, customers and dismantlers respectively.
To explain the proposed algorithm clearly, a numerical example is provided.In this example, I = 3, J = 5, K = 2 which means that three suppliers provide materials to five manufacturers and these five manufacturers provide finished products to two DCs.The chromosome at this stage in the example is shown in Figure 4 and the corresponding delivery route is shown in Figure 5a.In Figure 4, each gene holds a binary number, where 1 indicates the delivery route is adopted and 0 indicates it is not.Figure 4 shows that there has 15 genes for the first section of the chromosome.The first five genes illustrates the condition where supplier 1 provises materials to manufacturers 2 and 4 but does not provide materials to manufacturers 1, 3 and 5.The second five genes and the third five genes illustrate the related information of supplier 2 and supplier 3, respectively.Figure 4 also shows that there has 10 genes for the second section of the chromosome.The first two genes describe the condition where manufacturer 1 does not provide finished products to both two DCs.Similarly, the following four groups of genes describe the related information of the corresponding manufacturers.Figure 5a illustrates the delivery route.The numbers in parentheses means the unit transportation cost of each delivery route and the numbers in circles means the capacity of each partners.route.Since the chromosomes, in stage 1 only deliver the information of transportation route except for the quantity of product flow, it makes the genetic operations, especially the validation, much more concise than the traditional GA.
The chromosome possesses six sections corresponding to the six levels of the CLSC model.In each section, the amount of genes is equal to the product of the amount of suppliers and the amount of demanders.In general, the number of genes in a chromosome, is expressed by , where I, J, K, L and M are the number of suppliers, manufacturers, distribution centers, customers and dismantlers respectively.
To explain the proposed algorithm clearly, a numerical example is provided.In this example, I = 3, J = 5, K = 2 which means that three suppliers provide materials to five manufacturers and these five manufacturers provide finished products to two DCs.The chromosome at this stage in the example is shown in Figure 4 and the corresponding delivery route is shown in Figure 5a.In Figure 4, each gene holds a binary number, where 1 indicates the delivery route is adopted and 0 indicates it is not.Figure 4 shows that there has 15 genes for the first section of the chromosome.The first five genes illustrates the condition where supplier 1 provises materials to manufacturers 2 and 4 but does not provide materials to manufacturers 1, 3 and 5.The second five genes and the third five genes illustrate the related information of supplier 2 and supplier 3, respectively.Figure 4 also shows that there has 10 genes for the second section of the chromosome.The first two genes describe the condition where manufacturer 1 does not provide finished products to both two DCs.Similarly, the following four groups of genes describe the related information of the corresponding manufacturers.Figure 5a illustrates the delivery route.The numbers in parentheses means the unit transportation cost of each delivery route and the numbers in circles means the capacity of each partners.In Figure 4, each gene holds a binary number, where 1 indicates the delivery route is adopted and 0 indicates it is not.Figure 4 shows that there has 15 genes for the first section of the chromosome.The first five genes illustrates the condition where supplier 1 provises materials to manufacturers 2 and 4 but does not provide materials to manufacturers 1, 3 and 5.The second five genes and the third five genes illustrate the related information of supplier 2 and supplier 3, respectively.Figure 4 also shows that there has 10 genes for the second section of the chromosome.The first two genes describe the condition where manufacturer 1 does not provide finished products to both two DCs.Similarly, the following four groups of genes describe the related information of the corresponding manufacturers.Figure 5a illustrates the delivery route.The numbers in parentheses means the unit transportation cost of each delivery route and the numbers in circles means the capacity of each partners.

Stage 2-Freight Volume Decision
After the pairing, a delivery route is determined.Now, we have to decide the quantity for each agent needs to obtain.The second stage is to determine the freight volume of materials or products, and the chromosome encoded in the second stage is shown in Figure 6.

Stage 2-Freight Volume Decision
After the pairing, a delivery route is determined.Now, we have to decide the quantity for each agent needs to obtain.The second stage is to determine the freight volume of materials or products, and the chromosome encoded in the second stage is shown in Figure 6. Figure 6 show that the structure of the chromosome in stage 2 is the same as that in stage 1, but the composition of each gene is indeed different.The first five genes illustrates the condition where supplier 1 will deliver 50 units of raw materials to manufacture 2 and supplier 4, but will not delivery any raw materials to manufacturers 1, 3 and 5.The second five genes and the third five genes illustrate the related information of supplier 2 and supplier 3, respectively.The last two genes for the second section of the chromosome describe the condition where manufacturer 5 will deliver 200 units of products to distribution center 2 but will not deliver any products to distribution center 1.The network including delivery route and freight volume are shown in Figure 5b. Figure 6 show that the structure of the chromosome in stage 2 is the same as that in stage 1, but the composition of each gene is indeed different.The first five genes illustrates the condition where supplier 1 will deliver 50 units of raw materials to manufacture 2 and supplier 4, but will not delivery any raw materials to manufacturers 1, 3 and 5.The second five genes and the third five genes illustrate the related information of supplier 2 and supplier 3, respectively.The last two genes for the second section of the chromosome describe the condition where manufacturer 5 will deliver 200 units of products to distribution center 2 but will not deliver any products to distribution center 1.The network including delivery route and freight volume are shown in Figure 5b.

Process Outline
It is turned out that six steps are needed to determine the freight volume during freight volume decision process.To explain the freight volume decision process in detail, the numerical example in Section 4.1 is used.In this three stage supply chain example, the step details are illustrated in Figure 5 and described as below.The numbers in parentheses indicate the unit transportation cost of the delivery route.The numbers in the circles indicate the request of the distribution centers and the capacity of suppliers and manufacturers.
Step I: Determine the start level in the supply chain network.In this case, the terminal demand is at the distribution centers, hence the freight volume decision process begins at the level from manufacturer to distribution center.
Step II: Detect the top priority delivery route with the lowest unit shipping cost in the current level.
According to the result of "Cost Rank," the lowest unit shipping cost within current level is 3, which is from manufacturer 3 to distribution center 1.The detail of "Cost Rank" will be explained in Section 4.2.2.
Step III: Check if there are other delivery routes with the same unit shipping cost and also the same demand as the one found in Step 2. In other words, check if the top priority delivery route is multiple or not.If the result is yes, count out the number, mark as N and go to Step 4a, if not, go to Step 4b.
Step 4a and Step 4b are parallel.In this example, this unit shipping cost 3 is unique at this level, so go to Step 4b.
Step IVa: Devide the demand into N parts randomly and assigned them to the N delivery routes with the top priority.
Step IVb: Assign the smaller one to the transport flow through the comparison between the supply capability and the demand.Due to the supply capability of manufacturer 3 is 300 units and the demand of distribution center 1 is 500 units, the freight volume between them is 300 units.Step V: Renew the corresponding supply capability and demand.The remaining supply capability of manufacturer 3 is 300 − 300 = 0 and the renewed demand of distribution center 1 is 500 − 300 = 200.
Step VI: Move to the next delivery route on the basis of the result of "Cost Rank" and then repeat Step 2 to Step 6.When the entire level is completed, return to the next level.In this example, the delivery route between manufacturer 4 and distribution center 1 is found out according to "Cost Rank" as the next operand.
Specifically, when the unit transportation cost of several different suppliers delivering to the same demander is the same, the delivery flow will be randomly produced.In this case, since the unit transportation cost of both manufacturer 2 and manufacturer 5 to distribution center 2 are 6, the corresponding flows are randomly generated to be 50 and 200 respectively.

Order Bidding Strategy: Cost Priority Rule
At this stage, every agent will now be allocated with a certain quantity on hand.This quantity has to be supplied by its immediate up stream suppliers.To determine the winning supplier, we propose a Cost Priority Rule.In this case, each delivery route produces its own unit transportation cost.The route with low transportation costs will have priority for use, which is called cost priority rule.This cost priority rule is much proper in this problem than other rules such as demand priority and equal distribution.With this rule, the freight volume decision process can reach the edge value of the transportation cost in the whole CLSC under the pre-decided delivery route.Hence it can enhance the efficiency of the genetic search and increase the calculation speed.The flow allocation has to begin at the last stage of the proposed supply chain due to the imperative satisfaction of demand.It means the flow allocation begins from the level of distribution centers delivery to customers in the forward chain and then, move to the level of the manufacturers' delivery to the distribution centers.After that, rather than moving to the suppliers, it jumps to the reverse chain, which starts at the level of customers' recovery to the distribution centers and then, to dismantlers and manufacturers.The flow allocation of suppliers delivering to manufacturers is implemented as the end procedure, because the manufacturing demand depends on both the demand of the distribution centers in the forward chain and the supply quantity of the dismantlers in the reverse chain.

Genetic Operations
Genetic operations are indispensable parts in genetic algorithms.In the proposed problem, one-point crossover and one-point mutation is implemented to avoid dramatic changes in genetic structure due to the particular chromosome properties, which will prevent random genetic searches additionally.

Fitness Function
The fitness function is to calculate the fitness value of each chromosome which represents the viability of the chromosomes.In the proposed model, the fitness is assigned as reciprocal of the total cost.Lower total cost indicates stronger chromosome viability.

Selector Operator
In this proposed GA, after generating the initial pool randomly, generation of the mating pool is based on the roulette wheel selection.In this roulette wheel selection, the fitness function is used to decide the selected probability for each chromosome.The roulette wheel function is here p i means the selected probability for chromosome i, f i means the fitness of chromosome i.

Crossover
In this algorithm, one-point crossover is applied.The characteristic of one-point crossover is that genes within parent chromosome will not change greatly, therefore, after several evolution, weak genes will be identified easily [31].In this CLSC problem, the chromosomes contains several sections, each section represents one stage in the supply chain network.Since weak genes cause weak section, weak genes identification is crucial.This crossover process contains three steps as shown in Figure 7.
Step 1: Pick up two parents from the mating pool randomly.
Step 2: Randomly generate the cut point.
Step 3: Generate two offspring according to the cut point.

Crossover
In this algorithm, one-point crossover is applied.The characteristic of one-point crossover is that genes within parent chromosome will not change greatly, therefore, after several evolution, weak genes will be identified easily [31].In this CLSC problem, the chromosomes contains several sections, each section represents one stage in the supply chain network.Since weak genes cause weak section, weak genes identification is crucial.This crossover process contains three steps as shown in Figure 7.
Step 1: Pick up two parents from the mating pool randomly.
Step 2: Randomly generate the cut point.
Step 3: Generate two offspring according to the cut point.

Mutation
Due to the special feature of this two-stage encoding, the search of the proposed method is more easily trapped in a local optimum.Mutation process can prevent this situation.Therefore, one-point mutation is implemented, in which the mutation rate is set to 1.The process of mutation contains three steps as shown in Figure 8.

Mutation
Due to the special feature of this two-stage encoding, the search of the proposed method is more easily trapped in a local optimum.Mutation process can prevent this situation.Therefore, one-point mutation is implemented, in which the mutation rate is set to 1.The process of mutation contains three steps as shown in Figure 8.
Step 1: Pick up one chromosome from the mating pool.
Step 2: Randomly generate a mutation point and read the number of this gene as A.
Step 3: Calculate the result of 1 minus A and load this result to the gene of the selected mutation point.Step 1: Pick up one chromosome from the mating pool.
Step 2: Randomly generate a mutation point and read the number of this gene as A.
Step 3: Calculate the result of 1 minus A and load this result to the gene of the selected mutation point.

Validation
In order to guarantee the feasibility after crossover and mutation, validation is implemented.Due to the characteristics of two-stage encoding, the validation contains two parts.

Part 1: Check Capacity Requirements
There are several situations for offspring invalidation after crossover and mutation.In Part 1, the procedure checks on the binary chromosome, which is the chromosome before the "freight volume decision."If one of the situations in Table 1 occurs, rollback of crossover and mutation processes will be implemented.

Validation Requirements
1. Active distribution centers must have enough capacity to satisfy customers demand totally.2. Active manufacturers must have enough capacity to satisfy customers demand totally.3. Since each customer must recovery at least one return product to distribution centers, the customer genes are forbidden to be zero simultaneously.4. Customer must have at least one corresponding distribution center due to the satisfaction need of customers.Take Figure 7 for example, the active manufacturers in offspring 1 is M3, M4 and M5.In this case, the total capacity can't satisfy the total customer demand, which causes invalidation to rollback the whole crossover process.The selected parents chromosome will be assigned as the offspring after crossover.
Part 2: Check Penalty If the offspring get through the capacity check in Part 1, the penalty check in Part 2 will be implemented.In Part 2, this validation procedure acts on the integer chromosome, which is the chromosome after the "freight volume decision."In this step, the penalty for the transformed chromosome is examined.If the penalty is not equal to zero as before, the procedure will undo the whole process of respective crossover and mutation.
In Figure 8, offspring 2 after mutation shows the invalidation of the penalty in Part 2. To show the detail and explain the calculation process of the penalty, the network of offspring 2 is drawn in Figure 9.
Mathematics 2020, 8, x FOR PEER REVIEW 13 of 20 chromosome after the "freight volume decision."In this step, the penalty for the transformed chromosome is examined.If the penalty is not equal to zero as before, the procedure will undo the whole process of respective crossover and mutation.
In Figure 8, offspring 2 after mutation shows the invalidation of the penalty in Part 2. To show the detail and explain the calculation process of the penalty, the network of offspring 2 is drawn in Figure 9.

Stopping Condition
The proposed approach will improve the initial pool iteratively until one of the stopping conditions below is met: a preset maximum number of generations is reached or a feasible solution is generated in twenty successive generations.

Stopping Condition
The proposed approach will improve the initial pool iteratively until one of the stopping conditions below is met: a preset maximum number of generations is reached or a feasible solution is generated in twenty successive generations.

Computational Experiments
In this section, three computational experiments are implemented to demonstrate the efficiency and stability of the proposed approach.For the convenience of comparison, the same dataset as Wang and Hsu's paper [17] is used for Experiment 1.In this referenced paper, a revised spanning-tree based GA was proposed to optimize the CLSC model.The great difference between the fixed costs of the entities in Experiment 1 shows that it is an extreme case in this computational experiment.Hence, Experiment 2 and Experiment 3 are needed to illustrate more usual cases.Based on existing literature, such as Atabaki et al. [22] and Min et al. [28], we make some changes and improvements in the following experiments.Their exist similar values within the capacity and fixed cost of distribution centers in Experiment 2, which make the optimal search become difficult.In Experiment 3, the fixed cost and the corresponding capacity within same level entities are set to be even more closer to make the optimal search even more difficult.All of the three experiments include five sub-problems with a scale from basic to large.Table 2 shows the five scales.

Experiment 1
In this experiment, the dataset in Wang and Hsu's paper [17] has been used.With the same input data, the comparison between the single-stage encoding spanning tree based GA and the proposed GA is convincing.
After computing the proposed two stage priority and multi-agent based GA, the result of the basic scale problem turns to be uniform with the referenced paper.In other words, with the uniform of the total costs, their exist slightly differents in the flow volume between each level.This result indicates the acquirement of another optimal solution for this NP-hard problem.
To further test the proposed approach, four larger scale problems have been computed.The dataset is also the same as the original paper.Table 3 shows the comparison between the proposed approach and the revised spanning tree based GA (Revised ST-GA) in Wang and Hsu's paper [17].Table 4 shows the comparison between the proposed GA and Lingo.The results of Lingo and the provided GA are worked out using a HP PC with Intel(R) Core(TM) i7-2600 CPU @ 3.4 GHz, 8.0 G RAM.The result of the referenced paper was obtained using a PC with Intel ® Pentium ® M processor 1.86 GHz, 1.0 G RAM.In Table 3, the results of revised ST-GA are from the original paper.Suite reference again, from the optimal value comparison and the average time comparison, it is obvious that the proposed approach can produce higher quality results within a shorter computing time.Take Scale 2 for example, in the optimal value comparison, the percentage difference is 0.07%, which means the result of the proposed approach is 0.07% better than the spanning tree GA.Furthermore, in the average time comparison, the percentage difference is 80.9%, which means the running time of the proposed approach is 80.9% faster than the spanning tree GA.When the scale of problem increases, the accuracy of the proposed approach is improving and the running time is always shorter.In Scale 3, the result of the proposed approach is 0.54% better than the spanning tree GA and the average running time is 58.9% better.In Scale 4, the result of the proposed approach increases to 0.99% better with 44.7% of the running time, which is twice as fast as the speed of the spanning tree GA.Furthermore, in Scale 5, when the number of variables is almost 19,000, the quality result is 1.27% better than the spanning tree GA with about one third of the running time 35.2%.
Table 4 indicates the comparison results of Lingo 11.0 and the proposed approach with five scales.The "absolute difference" row shows the value of the Lingo values minus the results of the proposed approach.The minus sign means disparity.For instance, in Scale 2, the absolute difference value is −19, the value of percentage difference is −0.03%, which indicates the disadvantageous of the proposed approach is 0.03% compared to Lingo.However, the average running time of the proposed approach is 1.21 s, which is only 10.1% of the time from Lingo.When the scale of the problem grows larger, the speed advantage of proposed approach is more obvious.In Scale 3, the disadvantageous of the proposed approach is 0.38% compared to Lingo, but the running time is less than 0.93% compared with Lingo.In Scale 4, the disadvantageous of the proposed approach is 2.14% compared to Lingo, but the time of running is less than 1.49%.In Scale 5, while the performance of the proposed approach is 1.77% behind Lingo, the running time is better than Lingo by almost 40 times.Furthermore, if the scale of problems keeps growing, Lingo cannot even give out an optimal result within an acceptable time.However, the proposed approach can solve it effectively.
In this basic scale model, the landfilling rate is set to 0.1 currently.To test the robustness of the results and study the influence of the landfilling rate on the total cost, we provide a sensitivity analysis.
Since it is an integer programming, which is discrete, we illustrate an arithmetic sequence in the range of landfilling rate which is from 0 to 1.Where 0 means no landfilling and 1 means landfilling all without recycling.Table 5 shows the calculation results of the landfilling rate and the corresponding total cost.Figure 10 shows the line graph of the relationship between landfilling rate and the total cost.From Table 5 and Figure 10, it can be seen that the total cost increases slightly with the increase of the landfilling rate and remains basically constant, which indicates that the model has a good robustness.approach is 1.77% behind Lingo, the running time is better than Lingo by almost 40 times.Furthermore, if the scale of problems keeps growing, Lingo cannot even give out an optimal result within an acceptable time.However, the proposed approach can solve it effectively.
In this basic scale model, the landfilling rate is set to 0.1 currently.To test the robustness of the results and study the influence of the landfilling rate on the total cost, we provide a sensitivity analysis.Since it is an integer programming, which is discrete, we illustrate an arithmetic sequence in the range of landfilling rate which is from 0 to 1.Where 0 means no landfilling and 1 means landfilling all without recycling.Table 5 shows the calculation results of the landfilling rate and the corresponding total cost.Figure 10 shows the line graph of the relationship between landfilling rate and the total cost.From Table 5 and Figure 10, it can be seen that the total cost increases slightly with the increase of the landfilling rate and remains basically constant, which indicates that the model has a good robustness.

Experiment 2
The results of Experiment 1 show that, manufacturer 1, manufacturer 4 and distribution center 3 are not used due to their high fixed cost and low capacity.To make the model more complex,

Experiment 2
The results of Experiment 1 show that, manufacturer 1, manufacturer 4 and distribution center 3 are not used due to their high fixed cost and low capacity.To make the model more complex, manufactures fixed cost and distribution centers fixed cost are reset.To release more active entities, the unit shipping costs are reset too.After the data changing, the parameters within distribution centers become similar, which tight the optimal solution to make the optimal search more difficult.Table 6 shows the capacity and fixed cost.Table 7 indicates the unit shipping costs.Since the dataset is different to that in the original paper [17], the comparison is only between the proposed approach and Lingo.For simplicity, only the dataset of the basic scale is shown.The other four scales are doubled and redoubled to the basic scale, which was the same data building method as that in the original paper [17].The optimal solution is shown in Table 8.Table 9 is the comparison with Lingo.Since Lingo is an exact algorithm and GA is a heuristic algorithm, it is usual that Lingo can get better results when the calculation scale is relatively small.But when the calculation scale gets large, Lingo may cannot get an acceptable result within an acceptable period of time.That's where heuristic algorithm works.
From Table 9, it can be seen that although the quality of results is reduced, the calculation speed is far faster than Lingo.In Scale 1, the proposed approach can find the same optimal solution as Lingo within a shorter time, which is only 15% of the Lingo running time.In Scale 2, the disadvantageous is 0.21% compare to Lingo, but the time of running is only 28%.When the scale grows to Scale 3, the result is 0.94% poorer than Lingo but the running time shorts to less than 0.94% of it.In Scale 4, the disadvantageous is 3.37% compare to Lingo, however the running time shortens to 2.72%.In Scale 5, the disadvantageous becomes 2.5% compare to Lingo, while the time of running is less than 2.87%.
It is observed from the "Percentage difference" row that all of the disadvantageous are below 3.5%, however, the running time in the "Percentage of time" row are 100 times faster than Lingo.

Experiment 3
In order to further tight the optimal solution of the whole network to add difficulties in the searching, the parameters are further modified.From the optimal solution of the basic scale in Experiment 3 shown in Table 10, it can be seen that four manufacturers and three distribution centers are being used.
For clarity, only the changed part of the unit shipping cost is shown, the other part is the same as that in Experiment 2. Table 11 shows the capacity and fixed cost.Table 12 shows the unit shipping cost for each stage.The optimal solution is shown in Table 10.Table 13 is the comparison with Lingo.Table 13 shows the results and comparison with Lingo.In Scale 1, the proposed approach can obtain the optimal solution within just 12% of the running time of Lingo.In Scale 2, the disparity is 0.44% to Lingo but the running time is 15.6%.When the problem scale grows to Scale 3, the result is a 1.73% disadvantage to that of Lingo but the time is less than 0.9%.In Scale 4, the disadvantageous is 4.7% compare to Lingo, but the running time is less than 1.83%.In Scale 5, the disparity is 3.86% to Lingo, however, the running time is less than 2.97%.In Experiment 3, the optimal search becomes more difficult, while the error rate is still below 5%, which can be checked from the "Percentage difference" row.The "Percentage of time" row indicates the obvious advantage of the running time performance.From the results of Experiment 3, it can be further demonstrated that when the scale of the problem grows, the proposed approach can still give a near optimal solution with a fast calculation time.

Conclusions and Suggestions of Future Work
Due to growing environmental issues nowadays, product recycling problems within closed-loop supply chain draws more attention.To solve these problems, many CLSC models have been established in this research area.A six-level CLSC model studied in this paper can be transformed into an integer linear programming model.The small-scale calculation of this model can be solved by using LINGO.However, LINGO cannot solve the large-scale model in an acceptable time.This is due to that the complexity of the model will increase as the scale increases.A spanning-tree based GA is proposed by Wang and Hsu [17] in 2010 so as to solve the problem.However, the single-stage encoding method cannot reach a fully exploit of the search ability of GA.In this study, a new two-stage priority-based GA encoding with multi-agent has been established to improve the solution of this kind of CLSC model.It helps to the research of implementing GA more efficiently on the CLSC problem.
Three computational experiments have been implemented.Experiment 1 has the same dataset as Wang and Hsu's experiment and the results show that the proposed algorithm can reliably obtain higher quality solutions in a shorter computation time.Since Experiment 1 is an extreme case with several entities idling, Experiment 2 and Experiment 3 are implemented to demonstrate the performance of the proposed approach in usual cases.In Experiment 2, all of the distribution centers are close in fixed cost and capacity which makes the optimal search more difficult.The results of Experiment 2 demonstrate that the proposed approach can get high quality results within shorter running times.Experiment 3 reset the parameters within the whole network to further tight the optimal solution, so as to make the optimal search even more difficult.The results of Experiment 3 show that the proposed approach still can provide a competent solution.Throughout these three computational experiments, the efficiency and stability of the proposed algorithm is proven.In the real case, the product flow in the CLSC network contains different kinds of products.In this sence, multi-product can be considered in future research.
forward and reverse flow in the Distribution Center (DC) k k pd the percentage of total capacity for reverse logistics in DC k in manufacturer j using materials from supplier i jk t transportation unit cost from each manufacturer j to each DC k kl u transportation unit cost from DC k to customer l km v transportation unit cost from DC k to dismantler m mj w transportation unit cost from dismantler m to manufacturer j lk Ru recovery unit cost in DC k from customer l j f fixed cost for operating manufacturer j k g fixed cost for operating DC k m h fixed cost for operating dismantler m  fixed cost for landfilling per unit

Figure 3 .
Figure 3.The flow diagram of two-stage priority based genetic algorithm (GA).Figure 3. The flow diagram of two-stage priority based genetic algorithm (GA).

Figure 3 .
Figure 3.The flow diagram of two-stage priority based genetic algorithm (GA).Figure 3. The flow diagram of two-stage priority based genetic algorithm (GA).

Figure 4 .Figure 5 .
Figure 4.The first two sections of chromosome in stage 1.

Figure 4 .
Figure 4.The first two sections of chromosome in stage 1.

Figure 4 .Figure 5 .
Figure 4.The first two sections of chromosome in stage 1.

Figure 5 .
Figure 5. (a) Delivery route of the numerical example.(b) Network of the numerical example.

Figure 6 .
Figure 6.The first and second sections of chromosome in stage 2.

Figure 6 .
Figure 6.The first and second sections of chromosome in stage 2.

Figure 7 .
Figure 7.The process of crossover.

Figure 7 .
Figure 7.The process of crossover.

Figure 8 .
Figure 8.The process of Mutation.

Figure 9 .
Figure 9.The network of offspring.

Figure 9 .
Figure 9.The network of offspring.

Figure 10 .
Figure 10.The relationship between landfilling rate and total cost.

Figure 10 .
Figure 10.The relationship between landfilling rate and total cost.

Table 2 .
Scale of computational experiments.

Table 3 .
Comparison of the proposed approach and the spanning tree based GA.

Table 4 .
Comparison between the proposed approach and Lingo.

Table 5 .
The relationship between landfilling rate and total cost.

Table 5 .
The relationship between landfilling rate and total cost.

Table 6 .
Capacity, fixed cost (US$) and demand in Experiment 2.

Table 7 .
Unit shipping cost for each stage (US$) in Experiment 2.

Table 9 .
Comparison with Lingo in Experiment 2.

Table 11 .
Capacity, fixed cost (US$) and demand in Experiment 3.

Table 12 .
Unit shipping cost for each stage (US$) in Experiment 3.

Table 13 .
Comparison with Lingo in Experiment 3.