Optimal Load Dispatch in Competitive Electricity Market by Using Di ﬀ erent Models of Hopﬁeld Lagrange Network

: In this paper, a Hopﬁeld Lagrange network (HLN) method is applied to solve the optimal load dispatch (OLD) problem under the concern of the competitive electric market. The duty of the HLN is to determine optimal active power output of thermal generating units in the aim of maximizing the beneﬁt of electricity generation from all available units. In addition, the performance of the HLN is also tested by using ﬁve di ﬀ erent functions consisting of the logistic, hyperbolic tangent, Gompertz, error


Introduction
Optimal load dispatch (OLD) is a traditional optimization problem in electric power system operation with the duty of allocating the best active power output of operating thermal units so that the total electricity generation fuel cost is reduced as much as possible [1]. The concerned OLD problem has attracted a huge number of researchers so far, and there has been a vast number of applied optimization methods such as particle swarm optimization (PSO) [2], differential evolution (DE) [3], the genetic algorithm (GA) [4], the cuckoo search algorithm (CSA) [5,6] and other state-of-the-art Energies 2019, 12, 2932 2 of 24 methods [7][8][9][10]. In connection with optimization algorithms, these studies have focused on applying new algorithms and improving these original ones for finding valid solutions with high quality and satisfying all constraints. In connection with the objective function complexity, different models of fuel cost function related to single fuel, multiple fuels and valve point loading effects have been taken into account. On the other hand, constraints regarding thermal generating units as power output limits, ramp rate limits, and prohibited zone, as well as constraints regarding power systems such as spinning reserve and power balance of power systems were considered seriously. In order to evaluate the effectiveness and robustness of the applied methods, comparisons of fuel cost and simulation time have been investigated.
Obviously, the OLD problem has a very important role in operation field; however, the problem can become more important if the current competitive electric market can be taken into account [11][12][13]. In the competitive market, electric power providers must supply electricity to their customers with low electricity prices. This also means the customers can maximize their profits by choosing the most appropriate provider [14]. However, in the electric market, the energy providers must undertake two issues. The first issue is to determine a provision of energy that will be sold to load in coming hours, and the second issue is to calculate how much energy should be sold and should be reserved for future so that the profit can be maximized [15]. The competitive electric market has been considered in the unit commitment problem and studied in [16][17][18][19][20][21][22][23][24][25][26], consisting of different methods such as the hybrid Lagrange relaxation and evolutionary programming (HLR-EP) [16], the muller method (MM) [17], the tabu search based hybrid optimization technique (HTS) [18], the memetic algorithm (MA) [19], parallel artificial bee colony (PABC) [20], nodal ant colony optimization (NACO) [21], the multi-agent modeling method (MAMM) [22], the binary fish swarm algorithm (BFSA) [23], hybrid LR-secant invasive weed optimization (HLR-SIWO) [24], the sine cosine algorithm (SCA) [25] and the binary whale optimization algorithm (BWOA) [26]. In addition, the OLD problem has been applied under the consideration of competitive environment [27][28][29][30][31].
The Lagrange Hopfield network (HLN) improves on the Hopfield neuron network by combining the energy function and the Hopfield network in order to reduce the oscillations in converging to optimal solutions with very tiny errors [32]. In the HLN, the Lagrange optimization function is first built and is then converted to the energy function with the presence of the outputs of continuous neurons and multiplier neurons. The strategy for solving optimization problems by implementing the HLN consists of update processes such as the updating of dynamics of inputs and outputs for multiplier neurons and for continuous neurons, the updating of inputs for multiplier neurons, the updating of inputs for continuous neurons, and the updating of outputs of continuous neurons. Among the update processes, the last update is used to directly calculate optimal solutions. The HLN was successfully applied in 2012 [32] for dealing with the economic load dispatch problem without considering the competitive electric market. Its results were better than almost all compared methods in terms of fuel cost, convergence time and very tiny errors. The HLN in [32] solved the OLD problem with two cases, the first of which considered all thermal units and the second of which took both hydropower plants and thermal power plants into account. Numerical results and graphic results have led to the conclusion that the HLN could deal with large-scale systems and complicated constraints easily without oscillations like the Hopfield neuron network.
In [33], the augmented Lagrange Hopfield network (ALHN) was developed for solving the OLD problem by considering the electrical market. The method is an expanded form of the HLN, since the Lagrange optimization function is expanded into the augmented Lagrange function with the presence of equality constraints. The method has shown better profit than PSO and DE for two power systems with three units and ten units. However, the study did not clearly point out the real performance of the ALHN since initial outputs were fixed at the same values, and result comparisons with the HLN have not been accomplished. Furthermore, the ALHN is more complicated than the HLN due to the presence of a higher number of Lagrange multipliers. Both the HLN in [32] and the ALHN in [33] employed the sigmoid function for updating continuous neurons. Consequently, in this paper, we propose to use the HLN with five different functions for updating outputs for continuous neurons. In order to evaluate the performance of the HLN, we also implement such five HLN methods for solving two systems with three units and ten units. The novelties and the contributions of the paper can be summarized as follows: (1) First apply the HLN to the OLD problem while considering the electric market.
(2) Propose five different functions for updating outputs for continuous neurons.
(3) Use different initial outputs for continuous neurons for evaluating the oscillations of the HLN.
In addition, the main contributions of the paper can be summarized as follows: (1) Reduce the complicated level of the ALHN in establishing energy function.
(2) Reduce control parameters by canceling the augmented terms in the Lagrange function. This can shorten simulation time.
(3) Point out the best function for updating outputs for continuous neurons. The best function can stabilize the search performance of the HLN. (4) Survey the oscillations of the HLN by different initial outputs for continuous neurons. (5) Five functions form five HLN methods, and their results from two systems with three units and 10 units will be compared to those of other methods such as the cuckoo search algorithm (CSA), particle swarm optimization (PSO), differential evolution (DE) and the ALHN.
The remaining parts of the papers are as follows: The problem formulation with an explanation of objective function and constraints is given in Section 2. An implementation of HLN methods for solving the considered problem is described in detail in Section 3. Two test systems with three units and 10 units are solved for comparison in Section 4. Section 5 summarizes the conclusions of the work.

Objective Function
The OLD problem in the competitive electric market is established by the presence of an objective function and a set of constraints regarding thermal generating units as well as power systems. In order to present the considered objective function, the fuel cost function for generating electricity is first mentioned as follows: However, all thermal generating units must generate higher than the requested demand due to reserve power demand (PR i ) during operation in the market. As such, the sum of P i and PR i leads to a higher cost, which is calculated as follows: Considering the probability (P a ) for the reserve required and produced, the total fuel cost is obtained by [16]: As power companies sell electric energy to customers, revenue (RE) can be calculated by using the two following models:

1.
Payment for delivered power As a result, total profit (TP) can be obtained using RE and TFC and is also the considered objective, as defined by: In the HLN, objective function should be minimized. As such, the objective above is also similar to the objective below

The Set of Constraints
In addition to the objective function, a set of constraints must be taken into account, and they must be satisfied as follows: 1.
The active power balance between demand and supply: The total generation of all units and load demand P D must follow the following rule:

2.
Active power reserve constraint: The sum of reserve power from all units and the reserve demand PR D are constrained by the following inequality: 3. Generation limits: The power output of each thermal generating unit must be within the lower bound P min i and the upper bound P max i as the following model: The constraint aims to assure the safety of the generator while producing electricity. Normally, each thermal generating unit does not have a lower bound subject to physical ability, but it must be constrained by the lower bound due to economic issues [34]. During the operation of thermal generating units, the fuel cost for starting up each thermal generating is significant. Thus, it must be worked with large enough power to avoid high fuel cost.

4.
Reserve limits: The active power reserve of the ith unit PR i must follow the rule below: In the two equations above, PR i is the power reserve of the ith thermal generating unit, and it is not constrained by a specific value. However, its maximum value PR max i must not be higher than (P max i − P min i ). However, the sum of the power reserve of all thermal generating units must satisfy Constraint (9) above. As Constraints (8)- (12) are exactly met, the power system can work stably and safely.

Implementation of the HLN for the OLD Problem in the Competitive Electric Market
The HLN can deal with the OLD problem in the electric market or other optimization problems by establishing the Lagrange function and energy function together with processes of neurons. The main structure of the HLN can be summarized as follows: 1.
Establish the Lagrange function: The Lagrange function must include objective functions and constraints in which each constraint will have one Lagrange multiplier that needs to be tuned for optimal solutions that satisfy all constraints and have a high quality [35].

2.
Establish the energy function: The energy function is a converted function from the Lagrange function. Here, the control variables and the Lagrange multiplier in the Lagrange function will become outputs for continuous neurons and multiplier neurons, respectively. In addition, the inverse sigmoid function is also added in the energy function [33].

3.
Calculate the dynamics of neurons: The dynamics of neurons can be determined by taking partial derivatives of energy function with respect to the outputs for continuous neurons and multiplier neurons.

4.
Update inputs for multiplier neurons and continuous neurons: Inputs for neurons must be updated after determining the dynamics of neurons by adding a change step to old inputs. The change step is calculated from the dynamics of neurons.

5.
Update outputs for multiplier neurons and continuous neurons: The updated outputs for multiplier neurons are used to calculate dynamics of neurons in next iteration. Meanwhile, the updated outputs for continuous neurons are control variables that are added in an optimal solution if all termination conditions are exactly met as expected.
In addition to the five computation steps, the HLN needs initial parameters for starting the first iteration and needs termination conditions. Randomization is used to produce initial parameters such as inputs and outputs for both multiplier.

Lagrange Optimization Function and Energy Function
The HLN is carried out for optimizing the objective function and handling all constraints exactly as in Section 2. The first main step of the HLN is to construct the Lagrange optimization function, and then the Lagrange function is converted into the energy function. The Lagrange function, consisting of the objective function and constraints, can be mathematically formulated as follows: In Equation (13), N i=1 P i − P D is taken from power balance constraint in Equation (8); PR i − PR D is taken from reserve power constraint in Equation (9); and P i + PR i − P max i is taken from reserve limit constraints in Equation (12). In addition, Sign P , Sign i , PR and Sign PR are the signs of the three term above and can be determined by the following equations: Energies 2019, 12, 2932 6 of 24 The Lagrange function (13) can be transferred into the energy function by converting the control variables and the Lagrange multiplier into outputs for neurons. In addition, the inverse sigmoid function is also added in the energy function. As a result, the energy function is formed as follows:

Dynamics of Neurons
In order to update outputs as well as inputs for neurons, the dynamics of neurons must be first updated based on the models below:

Update Inputs for Neurons
The inputs of the continuous neurons and the Lagrange multiplier neurons at the current iteration can be updated by: where α 1 , α 2 , α 3 , α 4 and α 5 are positive scaling factors and do not have a specific range like P a or other parameters. As such, the most appropriate values are obtained by experiment. This issue is the main shortcoming of the HLN in dealing with optimization problem, especially for complicated problems with a high number of constraints and high number of control variables [34]. However, the complexity of the HLN can be reduced thanks to the simplicity of updating outputs for multiplier neurons, since the outputs for multiplier neurons can be set to the input for multiplier neurons. The setting can also lead to good results, so more solutions for finding the parameters are no longer required. The outputs are determined by:

Update Output for Neurons
The outputs for continuous neurons are updated by using the following models: where five functions consisting of the hyperbolic tangent function, the logistic function, the gom function, the erf function and the gd function can be defined as follows: In [32], only one function, tanh(σU i ) (where σ is the slope and U i is the input of neurons), was used to determine output for continuous neurons. There were three curves plotted in [32] corresponding to three values of σ, which were 0.005, 0.01 and 100. In this paper, we used the function together with four other functions shown in Equations (43)-(47). As such, five curves are plotted in Figure 1 that correspond to the five functions in which x is varied from −π to π.
( ) max min , min , where five functions consisting of the hyperbolic tangent function, the logistic function, the gom function, the erf function and the gd function can be defined as follows: In [32], only one function, tanh(σUi) (where σ is the slope and Ui is the input of neurons), was used to determine output for continuous neurons. There were three curves plotted in [32] corresponding to three values of σ, which were 0.005, 0.01 and 100. In this paper, we used the function together with four other functions shown in Equations (43)-(47). As such, five curves are plotted in Figure 1 that correspond to the five functions in which x is varied from −π to π.  Due to the use of five different functions, the five HLN methods are defined as the HLN-LF (the HLN with the logistic function), the HLN-THF (the HLN with the hyperbolic tangent function), the HLN-GF (the HLN with the Gompertz function), the HLN-EF (the HLN with the error function) and the HLN-GdF (the HLN with the Gudermanian function). It should be noted that each function contains both the slope and input of neurons, and each function is used to calculate the output for the neurons shown in Equations (33)-(42). Then, these outputs and inputs are used to calculate the dynamics of the neurons shown in Equations (18)- (22) in the next iterations. In next step, the dynamics of neurons are used to update inputs for neurons. The steps are repeated until the maximum error is not higher than Tol pre . Thus, the effectiveness of each function cannot be explained in theory, but using obtained results can illustrate their contribution to determine the output for neurons.

Selection of Parameters
In the HLN, there are many parameters need to be tuned. These parameters consist of (1) σ (2) α 1 , α 2 , α 3 , α 4 and α 5 (3) Predetermined tolerance Tol pre (4) Maximum iteration G max Among the parameters, σ directly influences the outputs for continuous neurons as shown in Equations (33)-(42). There is no predetermined range for σ; however, the parameter can be set to the same value of 100 for all study cases, and the results are good enough for acceptance. α 1 , α 2 , α 3 , α 4 and α 5 are set to small values which are higher than 0 but much smaller than 1. There is no rule for tuning the values of the five parameters excluding the trial and error method. For different study cases, these parameters are set to difference values. Tol pre has a high impact on the quality of optimal solutions, and it can be tried by setting to 10 −1 , 10 −2 , 10 −3 , 10 −4 and 10 −5 . When setting Tol pre to a very small value, the convergence is hardly ever reached. However, a higher value can easily lead to convergence, but the objective function of the obtained solutions, i.e., total profit, cannot reach the maximum value as expected. However, the impact of Tol pre on results when setting the range from 10 −5 to 10 −4 is insignificant-even the same. In contrast to the other parameters, G max does not lead to good or bad results, but it is employed to control the convergence of the HLN. In the HLN, the termination condition is based on maximum error. However, in order to avoid loss of control, G max can stop the iterative search process if the computational iteration is equal to G max . In this case, the termination condition is not exactly satisfied. If the maximum error is less than Tol pre , the computational iteration is smaller than G max . As such G max can be set to 5000 for all study cases.

Initialization
Inputs for both multiplier neurons and continuous neurons can be randomly produced within the range of 0-1. In addition, outputs for continuous neurons consisting of V i,P and V i,PR are randomly produced within the lower bound and the upper bound, while outputs for multiplier neurons consisting of V λ , V γ and V i,µ are randomly initialized between 0 and 1. The initialization can be summarized as follows: where ε 1 , ε 2 , ε 3 , ε 4 , ε 5 , ε 6 , ε 7 and ε 8 are random numbers within the range from 0 to 1.

Condition of Computation Termination
The search procedure ends if the current iteration (G) is equal to the maximum iteration (G max ) or the maximum error (Error max ) is equal or smaller than predetermined tolerance (Tol pre ). For all cases, we set G max = 5000 and Tol pre = 10 −4 , and Error max is determined by: Error max = max Error P , Error PR , max(Error PRi )

The Iterative Algorithm of the HLN for Dealing with the Considered Problem
The iterative algorithm of the HLN for solving the considered OLD problem in the competitive environment is given in Figure 2 and expressed by the following steps: Step 1: Set values for control parameters, as expressed in Section 3.2.1.
Step 2: Randomly generate inputs as well as outputs for multiplier neurons and continuous neurons, as shown in Section 3.2.2.
Step 3: Set current iteration G to 1.
Step 4: Determine the dynamics of inputs and outputs for all neurons, as shown in Section 3.1.2.
Step 5: Update the inputs for multiplier neurons and continuous neurons by using Section 3.1.3.
Step 6: Update the outputs for multiplier neurons and continuous neurons by using Section 3.1.4.
Step 7: Calculate individual error and maximum error, as shown in Section 3.2.2.
Step 8: If Error max > Tol pre and G < G max , set G = G + 1 and return to Step 3. Otherwise, stop the HLN and print results.

Condition of Computation Termination
The search procedure ends if the current iteration (G) is equal to the maximum iteration (Gmax) or the maximum error (Errormax) is equal or smaller than predetermined tolerance (Tolpre). For all cases, we set Gmax = 5000 and Tolpre = 10 −4 , and Errormax is determined by: The iterative algorithm of the HLN for solving the considered OLD problem in the competitive environment is given in Figure 2 and expressed by the following steps: Step 1: Set values for control parameters, as expressed in Section 3.2.1.
Step 2: Randomly generate inputs as well as outputs for multiplier neurons and continuous neurons, as shown in Section 3.2.2.
Step 3: Set current iteration G to 1.
Step 4: Determine the dynamics of inputs and outputs for all neurons, as shown in Section 3.1.2.
Step 5: Update the inputs for multiplier neurons and continuous neurons by using Section 3.1.3.
Step 6: Update the outputs for multiplier neurons and continuous neurons by using Section 3.1.4.
Step 7: Calculate individual error and maximum error, as shown in Section 3.2.2.
Step 8: If Errormax > Tolpre and G < Gmax, set G = G + 1 and return to Step 3. Otherwise, stop the HLN and print results.

Numerical Results
In the section, we have implemented the HLN with five functions for updating the outputs for continuous neurons, as shown in Equations (33)-(42). For each study case, we executed each HLN method for 100 independent trial runs, and then the results in terms of the average profit, maximum profit, minimum profit, average of iterations and maximum error were reported. The iterative algorithms were coded in Matlab program language and run on a 2.40 GHz personal computer.

Three-Unit System
In this section, the HLN method is tested on the three-unit system shown in Figure 3 with two revenue models consisting of payment for power delivered and payment for reserve allocated [33]. The two cases have the same input data such as the forecasted power demand of 1100 MW, the forecasted power reserve of 100 MW, the spot price of 11.3 ($/MWh) and the probability of reserve P a = 0.005. However, reserve price is different, i.e., three times the spot price for the first revenue model and 0.004 times the spot price for the second revenue model. For implementing PSO, DE and the CSA, we set the population and the maximum number of iterations to 5 and 500, respectively, while the other parameters for each method were set by the following selection:

Numerical Results
In the section, we have implemented the HLN with five functions for updating the outputs for continuous neurons, as shown in Equations (33)-(42). For each study case, we executed each HLN method for 100 independent trial runs, and then the results in terms of the average profit, maximum profit, minimum profit, average of iterations and maximum error were reported. The iterative algorithms were coded in Matlab program language and run on a 2.40 GHz personal computer.

Three-Unit System
In this section, the HLN method is tested on the three-unit system shown in Figure 3 with two revenue models consisting of payment for power delivered and payment for reserve allocated [33]. The two cases have the same input data such as the forecasted power demand of 1100 MW, the forecasted power reserve of 100 MW, the spot price of 11.3 ($/MWh) and the probability of reserve Pa = 0.005. However, reserve price is different, i.e., three times the spot price for the first revenue model and 0.004 times the spot price for the second revenue model. For implementing PSO, DE and the CSA, we set the population and the maximum number of iterations to 5 and 500, respectively, while the other parameters for each method were set by the following selection: For the first revenue model, the results obtained by five HLNs and the three methods together with the ALHN are reported in Table 1.  The obtained results from the five HLN methods including maximum profit, mean profit, minimum profit, mean error, and mean iterations together with simulation time are shown in Tables  1 and 2 for the first and the second revenue models, respectively. In addition, results from PSO, the CSA, DE and the ALHN [33] are also reported in the tables for comparison. Table 1 indicates that all methods had the same maximum profit for the first revenue model with 1102.45 ($/h), while the mean For the first revenue model, the results obtained by five HLNs and the three methods together with the ALHN are reported in Table 1. The obtained results from the five HLN methods including maximum profit, mean profit, minimum profit, mean error, and mean iterations together with simulation time are shown in Tables 1 and 2 for the first and the second revenue models, respectively. In addition, results from PSO, the CSA, DE and the ALHN [33] are also reported in the tables for comparison. Table 1 indicates that all methods had the same maximum profit for the first revenue model with 1102.45 ($/h), while the mean profit and the minimum profit of the proposed HLN were better than those of other methods excluding the ALHN because the ALHN used the same initial outputs for multiplier neurons and continuous neurons. These results mean that the five HLNs had the best stability, and approximately all runs had the same profit as the best run. In addition, the five HLN methods were also faster than other ones, since the mean iterations were from 40 to 161 and the simulation times of the HLN methods were the fastest from 0.017 to 0.069 s; the other methods used 500 iterations and took about 0.4-0.8 s. Similarly, the five HLN methods were better than other ones for the second revenue model. The maximum profit of all methods was approximately similar, but the mean and the minimum profits of the HLN methods were much higher. The observation of mean iterations and simulation times indicates that the HLN method could search optimal solutions much faster than other ones. Thus, it can be concluded that the HLN methods are the best methods among the applied methods. In comparison among the five HLN methods, it should be noted that the HLN-EF had the best performance because its mean error and mean iterations were the lowest. Its mean error was much smaller than Tol pre for two cases, but those of other methods were slightly smaller or higher than Tol pre . The mean errors were, respectively, 0.000078 and 0.000097 for the two cases, while Tol pre was equal to 0.0001. The mean iterations of the HLN-EF were 40 and 173, but those of other methods were from 59 to 161 for the first case and from 240 to 432 for the second case. Figures 4 and 5, respectively, show the maximum error and the profit at each computation iteration for the system with the first revenue model. These figures indicate that all HLN methods can stabilize the maximum error at each iteration since the fluctuations were nearly zero and the maximum error of the later iteration was lower than that of the previous iteration. For choosing the best method, the HLN-EF in red could reach the termination condition and the highest profit before the 40th iteration for the first model, whereas the other HLN methods were searching solutions, reducing the maximum errors and increasing total profits. The HLN-THF was the second best method for reaching the termination condition and the highest profit. Three remaining methods such as the HLN-GdF, the HLN-GF and the HLN-LF had the same manner in finding solutions, with about 140 iterations. The mean iterations shown in Table 1 also support this observation, since the three methods had the mean iterations higher than 150 iterations. Figures 6 and 7, respectively, illustrate the maximum error and the profit over the search process for the system with the second revenue model. The two figures also provide good evidence for indicating the superiority of the HLN-EF over other methods, since the method reached the smallest error at about iteration 170, while other ones used about approximately 250 and 450 iterations. As such, it can be concluded that error function is the best model for the updating outputs for continuous neurons.           All of the data of the system and optimal solutions obtained by the HLN-EF for the system are shown in Tables A1 and A2 of Appendix A.

Ten-Unit System
In this section, a 10-unit system which also had two revenue models was employed to run the HLN, PSO, DE and CSA methods. All of the data of the system were taken from [33]. The two cases had the same input data, such as a forecasted power demand of 1500 MW, a forecasted power reserve of 150 MW and a spot price of 31.65 ($/MWh). However, P a and PRP were different for two models-P a = 0.05 and PRP = 5 × PSP for the first model and P a = 0.005 and PRP = 0.01 × PSP for the second model. For implementing PSO, DE and the CSA, we set the population and the maximum number of iteration to 10 and 500, respectively, while the other parameters for each method were set to the same values shown in section above. The results from the HLN and other methods are shown in Tables 3  and 4 for two cases. For the first case, the maximum profit from the HLN methods was approximately equal to the best value of 14,564.731 $/h and was the highest value among all compared. The mean profit and the lowest profit of the HLN methods were nearly equal to the maximum profit and much higher than those from the PSO, CSA and DE methods. In fact, the highest profits of the three methods were, respectively, 14,182.186, 14,564.05 and 14,053.027 $/h. Meanwhile, their lowest profits were, respectively, 836.9154, 14,201.51, and 2281.539. Similarly, the maximum profit and the minimum profit of the proposed HLN methods were nearly alike and equaled 13,635.1081 $/h for the second case, but those of PSO, the CSA and DE were much worse. Their best profits were 13,158.0653, 13,635.105, and 13,093.1919 $/h, and their worst profits were 6246.4383, 13,177.6998, and 3729.7168 $/h, respectively. Furthermore, the execution time from the HLN methods was much shorter than those of the other methods. It was around 0.1 s for the HLN methods, but it was about 2.0 s for other ones, excluding the ALHN. Clearly, the HLN methods are much more effective for a large-scale system. Comparison among the five HLN methods had the same evaluation as the first system with three units. The HLN-EF was the best one with the highest maximum profit and minimum profit. In addition, the iteration from the HLN-EF was the smallest for two cases. Figure 8 show a whole view of search process for finding optimal solutions by using the first model of the revenue. As seen from this figure, these applied methods could not stabilize the maximum error over the whole search process. The maximum errors fluctuated from the 10th iteration to the 50th iteration, and then the fluctuation decreased and kept decreasing until the final iteration was reached. However, a clear view for observing the decrease of the maximum error cannot be presented in Figure 8. Therefore, Figure 9 was plotted for zooming in Figure 8 from the 120th iteration to the last iteration. Observing the five curves can see that the HLN-EF (in red) and the HLN-GF (in green) could reach the smallest fluctuations and the smallest maximum error; however, the HLN-EF was always better and reached the convergence first. The HLN-GdF and the HLN-LF (in black) were the two worst methods with the highest fluctuations and the highest maximum error. The HLN-THF (in blue) was separated into a group. Corresponding to the view of the maximum error, the view of the total profit can be seen by the meaning of Figures 10 and 11, where Figure 11 is plotted for zooming in Figure 10 from the 120th iteration to the last iteration. Figure 10 also has the same fluctuation as Figure 8 for the first fifty iterations, and the fluctuations decreased after the 50th iteration. However, the entire view of Figure 10 cannot show the clear stabilization of the total profit. Figure 11 indicates that the HLN-EF (in red) could reach the highest profit and stopped searching for new solutions at about the 180th iteration, while the HLN-GF (in green) found the second best profit after about two iterations. The three remaining methods still got a stable profit and reached maximum profit after the 200th iteration. Clearly, Figures 8-11 give good evidence for the outstanding performance of the HLN-EF over other ones for the first model. Similarly, the best performance of the HLN-EF for the second model of the system with 10 units can be observed by plotting Figures 12-15. The whole view of the maximum error and the total profit is presented in Figures 12 and 14, while Figures 13 and 15 zoom in Figures 12 and 14 for better views of search process. Figures 12 and 14 show the fluctuations of maximum error and total profit last from the first iteration to the 150th iteration; the fluctuations decreased dramatically until the last iteration was reached. Figures 13 and 15 point out the best performance of the HLN-EF in red and the second best performance of the HLN-THF in blue because the two methods stopped searching at about the 230th iteration with the highest profit, whereas other ones were still searching for solutions and increasing total profit. Consequently, it can be concluded that error function is the most appropriate function for updating the outputs for continuous neurons. were still searching for solutions and increasing total profit. Consequently, it can be concluded that error function is the most appropriate function for updating the outputs for continuous neurons.
All of the data of the system and optimal solutions obtained by the HLN-EF for the system are shown in Tables A3 and A4 of Appendix A.                All of the data of the system and optimal solutions obtained by the HLN-EF for the system are shown in Tables A3 and A4 of Appendix A.

Discussion on the HLN with Different Applied Functions
As can be seen in Tables 1 and 2 for the first system with three units, the HLN methods had approximately the same maximum profit. For the first model, all HLN methods could find the same profit of $1102.45, but for the second model, only the HLN-EF could find $1095.648-the other ones reached less profit. However, the deviation was not high. This means that all applied functions could support the HLN to find the best performance for the system. However, the number of iterations and the mean profit of all runs indicate that the HLN-EF is the best one because its mean and its maximum were equal. As seen from Tables 3 and 4, the phenomenon was nearly repeated for the second systems, and the HLN-EF was still the best method among the five HLN methods. However, the mean iterations from these study cases were different. The HLN-EF reached the smallest number of iterations (40 iterations) for the first model of the first system, while the mean iterations were higher for the second system because the second system was comprised of 10 units. In this study, we set many parameters to random values such as the inputs and outputs for the Lagrange multiplier neurons, and the power output and the power reserve of all units as shown in Equations (48)-(55). The setting and the simulation results mean that applications of the HLN methods are not dependent on the initial parameters that the ALHN in [33] suffered. By comparing all the HLN methods to PSO, DE and the CSA, it can be seen that the maximum profits indicated that the HLN methods were much more effective than these methods for the second system with 10 units, while the mean of profit over 100 runs indicated that the HLN methods were more stable in finding optimal solutions. This also shows that that the HLN methods were not influenced by the randomization factors that PSO, the CSA and DE had to suffer.

Conclusions
In this paper, we proposed five HLN methods for solving the OLD problem while considering the electricity market and complex constraints. Five functions consisting of the logistic, hyperbolic tangent, Gompertz, error and Gudermanian functions were employed to establish five HLN methods. Two systems with three and ten units were employed for two revenue models to run the HLN methods, with the CSA, PSO and DE methods being used for comparison. The comparisons of maximum profit indicated that the HLN methods could reach the same optimal solution as other

Discussion on the HLN with Different Applied Functions
As can be seen in Tables 1 and 2 for the first system with three units, the HLN methods had approximately the same maximum profit. For the first model, all HLN methods could find the same profit of $1102.45, but for the second model, only the HLN-EF could find $1095.648-the other ones reached less profit. However, the deviation was not high. This means that all applied functions could support the HLN to find the best performance for the system. However, the number of iterations and the mean profit of all runs indicate that the HLN-EF is the best one because its mean and its maximum were equal. As seen from Tables 3 and 4, the phenomenon was nearly repeated for the second systems, and the HLN-EF was still the best method among the five HLN methods. However, the mean iterations from these study cases were different. The HLN-EF reached the smallest number of iterations (40 iterations) for the first model of the first system, while the mean iterations were higher for the second system because the second system was comprised of 10 units. In this study, we set many parameters to random values such as the inputs and outputs for the Lagrange multiplier neurons, and the power output and the power reserve of all units as shown in Equations (48)-(55). The setting and the simulation results mean that applications of the HLN methods are not dependent on the initial parameters that the ALHN in [33] suffered. By comparing all the HLN methods to PSO, DE and the CSA, it can be seen that the maximum profits indicated that the HLN methods were much more effective than these methods for the second system with 10 units, while the mean of profit over 100 runs indicated that the HLN methods were more stable in finding optimal solutions. This also shows that that the HLN methods were not influenced by the randomization factors that PSO, the CSA and DE had to suffer.

Conclusions
In this paper, we proposed five HLN methods for solving the OLD problem while considering the electricity market and complex constraints. Five functions consisting of the logistic, hyperbolic tangent, Gompertz, error and Gudermanian functions were employed to establish five HLN methods. Two systems with three and ten units were employed for two revenue models to run the HLN methods, with the CSA, PSO and DE methods being used for comparison. The comparisons of maximum profit indicated that the HLN methods could reach the same optimal solution as other methods for the first system, but they could reach much better solution for larger system with ten units. The proposed HLN methods were also more stable and faster than other ones since they had a better mean profit, a better minimum profit, lower iterations and faster simulation times for the two systems. Thus, the HLN methods were superior to other compared methods. Among the five applied HLN methods, the HLN-EF with the use of error function was the best method, since the maximum profit, mean profit and minimum profit of all runs were approximately equal for two considered systems with two models of revenue. In addition, the HLN-EF reduced the maximum error and reached the highest profit fastest with the smallest number of iterations. Furthermore, the whole view of search process from the two systems indicated that the HLN-EF had the smallest fluctuations of maximum error and maximum profit, and it reached termination condition fastest. Consequently, it is recommended that the HLN and error function should be tried for other optimization problems in electrical engineering.
In this problem, with the consideration of the electric market, we have considered different prices for power produced and reserved from thermal generating units. Nowadays, electricity from solar or wind systems accounts for a high rate from all power sources [38,39]. As such, a consideration of renewable energies can be a good research field in optimizing load dispatch in the electric market. If all renewable energies can be joined in the electric market, power systems can become more stable and effective.

Conflicts of Interest:
The authors declare that there is no conflict of interests regarding the publication of this paper.

F i
Fuel cost of the ith thermal generating unit corresponding to power (P i + PR i ) λ Lagrange multiplier associated with active power balance constraint γ Lagrange multiplier associated with power reserve constraint of all available units µ i Lagrange multiplier associated with active power reserve constraint of the ith thermal unit PR min i , PR max i Minimum and maximum reserve power of the ith thermal unit P min i , P max i Minimum and maximum power output of the ith thermal unit a i , b i , c i Given cost function coefficients Error max Maximum error F i Fuel cost of the ith thermal generating unit corresponding to power output P i G Current iteration G c (V) Sigmoid function corresponding to output of neuron V G max Maximum iteration N Number of available thermal units P a Probability for power reserve required and produced P i Power output of the ith thermal unit PR i Reserve power of the ith thermal generating unit PSP, PRP Predicted sell price and predicted reserve price TFC Total fuel cost Tol pre Predetermined tolerance TP Total profit U λ , U γ , U i,µ Inputs for multiplier neurons U i,P , U i,r Inputs for continuous neurons V λ , V γ , V i,µ Outputs for Lagrange multiplier neurons V i,P , V i,PR Outputs for continuous neurons corresponding to power output and reserve power of the ith unit