An Optimization Model for Expired Drug Recycling Logistics Networks and Government Subsidy Policy Design Based on Tri-level Programming

In order to recycle and dispose of all people’s expired drugs, the government should design a subsidy policy to stimulate users to return their expired drugs, and drug-stores should take the responsibility of recycling expired drugs, in other words, to be recycling stations. For this purpose it is necessary for the government to select the right recycling stations and treatment stations to optimize the expired drug recycling logistics network and minimize the total costs of recycling and disposal. This paper establishes a tri-level programming model to study how the government can optimize an expired drug recycling logistics network and the appropriate subsidy policies. Furthermore, a Hybrid Genetic Simulated Annealing Algorithm (HGSAA) is proposed to search for the optimal solution of the model. An experiment is discussed to illustrate the good quality of the recycling logistics network and government subsides obtained by the HGSAA. The HGSAA is proven to have the ability to converge on the global optimal solution, and to act as an effective algorithm for solving the optimization problem of expired drug recycling logistics network and government subsidies.


Introduction
With the development of the economy, people pay more attention to their health. As a result, families keep more spare drugs in case of emergency, resulting in a dramatic increase in the number of expired drugs. In China, about 78.6 percent of families keep spare drugs, which produce 15,000 tons expired drugs per year [1]. If mistreated, expired drugs are very dangerous, as they can injure people's health if they carelessly take expired drugs, or badly damage the environment if they are thrown out without the right treatment. Therefore, how to recycle and treat expired drugs at the lowest cost has become an urgent problem that needs to be solved by the government [2].
Although there are lots of studies on recycling expired drugs, most of them are from a medical angle or about the status quo of recycling expired drugs [3,4]. Wang, Zhang and Zhang analyzed the dilemma faced by Chinese government in recycling expired drugs, and proposed government's responsibilities and obligations in this task, and put forward policies for the government [5]. Zhao and Ji analyzed the hazards of expired drugs and the status quo of drug recycling management in China, and proposed policies for expired drug disposal [6]. Liu and Qiu compared the effects of taxes and subsidies on expired drug recycling under the assumption that pharmaceutical manufacturing enterprises are responsible for recycling expired drugs, and found that subsidies are more economically reasonable and realistic [7].
There is some research on how to optimize expired drug recycling networks with the purpose of reducing operation costs. Kongar et al. proposed a reverse logistics framework that embodies environmental, economical, and physical concerns for end-of-life pharmaceutical products, and provided a radio frequency identification-based information technology infrastructure for the proposed system [8]. Kumar, Dieveney and Dieveney used the DMAIC process to analyze the pharmaceutical supply chain to improve the reverse logistics in a drug recall to avert the possibility of harm to consumers [9]. Narayana, Elias and Pati presented a systemic analysis of the complex interaction of factors affecting the reverse logistics process in a pharmaceutical supply chain, and found that there is strong linkage between the reverse logistics network design and key activities in returns management [10]. Kabir looked into detailed aspects of reverse logistics in the issues that pharmaceutical organizations face, and proposed that addressing the sustainability issue is beneficial to pharmaceutical organizations with end-of life products [11]. Amaro and Barbosa-Póvoa presented a modeling approach for the sequential planning and scheduling of supply chain structures with reverse flows, and applied it to the solution of a real case study of an industrial pharmaceutical supply chain [12].
However, no matter who are the executors of the recycling and disposal of expired drugs, pharmaceutical manufacturing enterprises or drugstores [13,14], and how optimal the expired drugs recycling logistics network is, the operational cost are so high that few executors have any interest in the process [9,15]. Even though the government can force enterprises to recycle and dispose of expired drugs by legislation or regulation [8], they may only recycle or dispose of part of the total volume of expired drugs due to the government's incomplete supervision. In addition, most of the expired drugs' owners are average consumers [16], and instead of sending back expired drugs, they may throw them away freely if they must bear the costs of returning expired drugs. Consequently, the result of expired drug recycling is not as good as expected, and it has been suspended in some regions of China [17].
Therefore, the most reasonable solution to the problem of recycling and disposing of expired drugs is that the government becomes the core of recycling expired drugs and drugstores become the executors of recycling expired drugs. In this solution, government takes the responsibility for designing the recycling logistics network to minimize the operation costs of recycling expired drugs, and optimizing the policies for subsidizing residents and drugstores to cover their costs incurred in recycling expired drugs. As a result, all the expired drugs should be recycled at the lowest total cost. This paper has two main contributions. Firstly, in this paper, the government is the core of expired drug recycling with the responsibility of designing the network, and uses a subsidy policy instead of just legislations or regulations as an incentive mechanism for the residents and drugstores. Secondly, it proposes a tri-level programming model to optimize the recycling logistics network and government subsidy policy, whereby people have the expired drugs, some drugstores are selected to take the responsibility of recycling expired drugs, and parts of the garbage disposal station network are selected to take responsibility for disposing of expired drugs, as a result, all of the expired drugs are recycled and disposed of, while the total costs are minimized.
The remainder of the paper is organized as follows: Section 2 is dedicated to the problem description and notations. The model is established in Section 3. We propose a HGSAA for the model in Section 4. An experiment is used to demonstrate the performance of the HGSAA in Section 5. Finally, conclusions are drawn in Section 6.

Problem Description and Notations
In order to reduce the family-owned expired drugs' damage to the environment or threats to people's health, a regional government decides to establish a recycling mechanism and networks to recycle residents' expired drugs. The target of the government's decision-making is recycling all the expired drugs at the lowest cost. Therefore, from N alternative drugstores in the region, the government selects n (n = 1,2,…,N) drugstores as recycling stations to recycle expired drugs from the residents in I residential sections. Meanwhile, from M alternative waste treatment stations, the government selects m (m = 1,2,…,M) waste treatment stations and upgrades them as expired drug treatment stations to dispose of expired drugs. The fixed costs and operation costs for each recycling station and treatment station are different. Their fixed costs are irrelative to the kind of expired drugs, while their operation costs vary according to the kind of expired drugs. The residents sending expired drugs causes them certain costs, which are irrelative to the kind of expired drugs, while they vary directly with the distance from their homes to the recycling stations and the weight of the expired drugs. To encourage residents to send expired drugs to a recycling station, the government provides a certain amount of subsidies only in accordance with the number of expired drugs sent to the recycling station because the government is unable to know from where the residents send the expired drugs. As a matter of course, if the subsidies from government are more than the cost of sending expired drugs, people are willing to send them, or else, they refuse. Meanwhile the government gives the drugstores a certain amount of subsidies in accordance with the number of every kind of expired drugs recycled by them. As a matter of course, if the subsidies from the government are more than its recycling costs, the drugstore will be willing to be a recycling station, or else, it refuses. The government bears the freight costs of transporting expired drugs from recycling stations to treatment stations, and the fixed and operation costs of treatment stations.
The decision-making process of all participants is as follows: firstly, the government selects n drug stores as recycling stations, and m waste treatment stations as expired drug treatment stations, and establishes subsidy policies for residents and drugstores, in order to recycle all the expired drugs at the lowest cost. Then, targeting at maximizing the own profits, the selected drugstores decide whether to be a recycling station based on the government's subsidies and their costs. Finally, targeting at maximizing their own profits, the residents determine whether and where to send expired drugs based on government's subsidies and their freight cost. The notations used in this paper are as follows: Lij: the distance between residential section i, (i = 1,2,…,I) and alternative drugstore j, (j = 1,2,…,N). Ljk: the distance between alternative drugstore j (j = 1,2,…,N) and alternative waste treatment station k, (k = 1,2,…,M).
Dj: the decision of the j th (j ∈ Φn) selected drugstore on whether to be a recycling station, where the decision variable Dj = 0 means not to be a recycling station, Dj = 1 means to be a recycling station.
Rij: the decision of residents in residential section i, (i = 1,2,…,I) on whether and where to send expired drugs, Rij = 0 means sending no expired drugs to a recycling station j (j ∈ Φn), Rij = 1 means sending expired drugs to one of the recycling stations j-a decision variable.
sjk: the decision on whether to send expired drugs s (s = 1,2,…,S) from recycling station j (j ∈ Φn) to selected treatment station k (k ∈ Φm), sjk = 0 means sending no expired drugs s from recycling station j to the selected treatment station k, sjk = 1 means sending expired drugs s from recycling station j to selected treatment station k-a decision variable.
β1: the subsidies given to the residents per unit of expired drugs-a decision variable. β2: the subsidies given to the recycling stations per unit of expired drugs-a decision variable. γ1: the freight fee per expired drug per distance of the residents sending expired drugs. γ2: the freight fee per expired drugs per distance of sending expired drugs from the recycling station to the treatment station.

Model Development
In the decision-making process of recycling and treating expired drugs, the government is in the leadership, the drugstores and the residents are in the followership, that is, they make their decisions according to the government's policies. Firstly, the government makes the decisions on the set of recycling stations and treatment stations {Φn, Φm}, the routes for transporting the expired drugs from recycling stations to treatment stations {sjk}, and the subsidy policies {β1, β2}. Then, the selected drugstores decide whether to be a recycling station {Dj}. Finally, the residents determine whether and where to send expired drugs {Ri}. We will adopt backward induction to obtain their optimal decisions.
In expired drug recycling, the profits of residents in residential section i are: Obviously, the decision target of residents is maximizing their profits and making the maximum profit non-negative. Therefore, the residents are facing the following program, which is the bottom level model in our tri-level model: Solving (2) and (3), we can get the decision of resident i as: that is, the decision of resident i is the reaction function of the alternative recycling store j's decision Dj and the government's subsidy, β1. Like all information in this expired drug recycling scenario, the alternative recycling stores know the reaction function of resident i, and will make their decisions based on these reaction functions. Now, we can get the quantity of expired drug s (s = 1,2,…,S) sent to recycling station j (j ∈ Φn (Dj = 1)) is: The profits of recycling station j (j ∈ Φn (Dj = 1)) are: The decision target of recycling stations is maximizing their profits and making the maximum profit non-negative. Therefore, the recycling stations are facing the following program, which is the mid-level model in our tri-level model: Solving (7) and (8), we can get the decision of alternative recycling store j as: that is, the decision of alternative recycling store j is the reaction function of the government's subsidy β1.
Like all information in this expired drug recycling scenario, the government knows the reaction function of resident i and alternative recycling store j, and will make its decision based on these reaction functions.
Now, we can obtain the subsidy given to the residents as: The subsidy given to the recycling stations as: As the total quantity of expired drug s (s = 1,2,…,S) sent to recycling station k (k ∈ Φm) is: The total cost of expired drug treatment station k is: The total cost of all expired drug treatment stations is: The total freight fee of transporting expired drugs to the treatment stations is: Therefore, the total cost borne by government is: The decision target of government is recycling and treating all the expired drugs at the lowest cost. Therefore, the government is facing the following program, which is the upper level model in our tri-level model:

The Hybrid Genetic Simulated Annealing Algorithm
This multilevel programming problem is a typical NP-hard problem without a polynomial solution [18]. The genetic algorithm (GA) is widely used for NP-hard problems due to its outstanding capability of globally searching for the optimum. However, due to the GA's prematurity weakness, which can lead to the optimal solution found by the GA not being the real global optimal solution, many scholars adopt a hybrid genetic algorithm to solve multilevel programming problems [19]. Li et al. adopted a hybrid genetic algorithm to solve robust bi-level programming problems [20]. Li et al. proposed a hierarchical chaotic quantum-inspired genetic algorithm to solve a nonlinear bi-level programming problem, and verified the effectiveness of their algorithm [21].
Simulated annealing algorithm (SAA) is theoretically able to find the real global optimum, which remedies the shortcomings of the genetic algorithm [22,23]. Therefore, we adopt a hybrid genetic simulated annealing algorithm (HGSAA), whose particularities are converging more rapidly and obtaining the solution more accurately, to solve the tri-level programming of optimizing the recycling logistics network and subsidies for expired drugs.

The Algorithm Thought
The basic thought of the HGSAA in our paper is that GA is developed to rapidly search for an optimal or near-optimal solution among the solution space, and then SAA is utilized to seek a better one on the basis of that solution. Therefore, the disadvantages of genetic algorithms, which are prematurity and weak local searching capability, are effectively avoided, and the global and local search ability of the algorithm is enhanced. As a result, the global optimal solution is found rapidly.

Encoding
We use binary encoding in the HGSAA. If drugstore i or waste treatment station k are selected, their code will be set as 1; if drug store i or waste treatment station k are not selected, their code will be set as 0.

Fitness Function
As the objective function is minimizing the government's total cost, and the fitness function values must be nonnegative, so we define the fitness function as is the set of selected recycling stations and treatment stations, and TC (Φn, Φm) is the government's total cost under this set.

Selection
The selection method in the algorithm is roulette-wheel-selection. The greater the individual fitness value is, the more probable it is that the individual will be selected to be the next parent. The process of selection is as follows: firstly, calculate the selected probability Pi of the individual i,

Crossover and Mutation
Theoretically, the value of both the crossover probability pc and the mutation probability pm. ranges from 0 to 1. In common practice, pc ranges from 0.4 to 0.99, and pm ranges from 0.001 to 0.1 [24]. In this paper, we use sequencing crossover to exchange the sequence of the operations in the parent chromosomes with the crossover probability pc = 0.8, and use assignment mutation to change the assignment of a single operation in a single parent with the mutation probability pm = 0.05.

Simulated Annealing
In the genetic simulated annealing algorithm, through crossover and mutation, parent individuals p1 and p2 generate child individuals c1 and c2, which are accepted as the individuals of the next population with the probability P by calculating their fitness Fitp and Fitc. The acceptance probability P is shown as follows:

Termination or Convergence Criterion
If the algorithm satisfies the following criteria, then it terminates and outputs the best chromosome, that is, the optimal recycling logistic networks and subsidies: (i) The fitness value has no change after successive iterations, which means the current solution is the optimal solution. of the genetic algorithm, T0 is the start temperature, Te is the stop temperature, and μ is the annealing rate.

The Steps of the HGSAA
The steps of the HGSAA are as follows: Step 1 (Initialization). Get the encoding length, and set the population size S, the crossover probability pc, the mutation probability pm, the iteration number of the genetic algorithm I, the start temperature T0, the stop temperature Te, and the annealing rate μ.
Step 2. Generate the initial population p0 randomly, then evaluate the fitness function for the current population, and get the current best chromosome.
Step 3. Perform selection, crossover, and mutation operations on the current population to generate an offspring population p1.
Step 4. Perform a simulated annealing operation on the population p1 to generate a new population p2, then evaluate the fitness function for p2, and update the best chromosome.
Step 5. Terminate the process and output the best chromosome when the termination or convergence criterion is satisfied, otherwise, update the temperature and go to step 3 again.

Experimental Analysis
In this section, we use an arbitrarily chosen numerical experiment to illuminate our model. The government of one region plans to establish a logistics network and subsidy policy to recycle and treat all the expired drugs owned by the residents in this region. There are 15 residential sections, 10 drugstores (alternative recycling stations) and eight waste treatment stations (alternative expired drug treatment stations), whose location is shown in Figure 1. There are three kinds of expired drugs owned by the residents in this region. The average quantity of each kind of expired drugs in every resident is respectively α1 = 0.014, α2 = 0.02, and α3 = 0.016. The unit freight fee of the residents for sending the expired drugs is γ1 = 4, and that of transporting expired drugs from recycling stations to treatment stations is γ2 = 10. The distance between residential sections and drugstores is shown in Table 1. The distance between drugstores and waste treatment stations is shown in Table 2. The number of residents in every residential section is shown in Table 3. The fixed and operation costs of every alternative recycling station are shown in Tables 4 and 5        The parameters in our HGSAA are set as follows: the population size S = 100, the crossover probability pc = 0.8, the mutation probability pm = 0.05, the iteration number of the genetic algorithm I = 10, the start temperature T0 = 100, the stop temperature Te = 1, the annealing rate μ = 0.95. The result of HGSAA is shown in Table 8 and Figure 2. From Table 8 and Figure 2, we can find that the HGSAA found the optimal expired drugs recycling logistics networks and the optimal government's subsidy policies. Thus, it can be proved that the HGSAA is an effective algorithm for solving the optimization problem of recycling expired drugs.

Conclusions
In this paper, we have proposed a tri-level programming model to study how the government can design recycling logistics networks and subsidy policies to recycle and treat all the expired drugs owned by the residents at the lowest cost. In addition, combining the rapid global searching ability of a GA and the local searching ability of SAA, we proposed a HGSAA to search for the optimal solution of the model. Finally, an experiment was given to demonstrate the good performance of the HGSAA. It is found that the government can encourage the residents and drugstores to participate in expired drug recycling by optimizing the subsidy policies for them, and minimize the total costs of expired drug recycling by optimizing the expired drug recycling logistic network. As a result, all the expired drugs are recycled at the lowest total cost. Besides, it is proved that the HGSAA has the ability to converge on the global optimal solution rapidly and is an effective algorithm for solving the optimization problem of expired drug recycling and treatment policy for the government supply chain.