Hybrid GA-PSO Optimization of Artiﬁcial Neural Network for Forecasting Electricity Demand

: In the present study Artiﬁcial Neural Network (ANN) has been optimized using a hybrid algorithm of Genetic Algorithm (GA) and Particle Swarm Optimization (PSO). The hybrid GA-PSO algorithm has been used to improve the estimation of electricity demand of the state of Tamil Nadu in India. The ANN-GA-PSO model uses gross domestic product (GSDP); electricity consumption per capita; income growth rate and consumer price index (CPI) as predictors that affect the electricity demand. Using the historical demand data of 25 years from 1991 till 2015 it is found that ANN-GA-PSO models have higher accuracy and performance reliability than single optimization models such as ANN-PSO or ANN-GA. In addition, the paper also forecasts the electricity demand of the state based on “as-it-is” scenario and the scenario based on milestones set by the “Vision-2023” document of the state.


Introduction
Electricity reforms have liberalized the electricity sector in many countries.The salient features have been unbundling of generation, transmission and distributions entities; a competitive market with in countries and creation of an independent regulator for access to transmission infrastructure.
In the prevailing deregulated markets, forecasting of electricity demand has emerged as a key research field [1][2][3].Many research tools and algorithms have been developed for electricity demand forecasting.Most of modeling techniques fall under parametric or non-parametric categories.Parametric techniques [4][5][6][7][8] are incapable of adapting to any type of environmental or societal changes.Many parametric techniques such as Auto regressive Integrated Moving Average (ARIMA), Exponential technique and Multiple Linear Regression when used for electricity demand forecasting do not yield the desired accuracy [9].In order to overcome the respective drawbacks of the parametric techniques and to provide the ability of global search non-parametric (artificial intelligence) techniques are preferred by researchers [10][11][12].
Artificial Neural Network (ANN) is very popular amongst researchers due to its adaptability over wide range of problems involving decision making in uncertain situations.This has led to the rapid developments of hybrid models [13].Many variants of ANN involving hybridization by learning techniques such as Backward Propagation (BP), Genetic Algorithm (GA) and Particle Swarm Optimization (PSO) have been proposed by several researchers.The use of ANN with different optimization methods is also useful to forecast the electricity demand.Amjadi, N. and Keynia Farshid [14] presented a stochastic search technique based on hybridization of ANN for load forecasting problem.According to them the hybridized ANN algorithm allows effective search of the solution space without falling in local minima.Abdul, H. et al. [15] also came up with similar conclusion about ANN model that was trained for short term load forecasting.Cincotti, S. et al. [16] has highlighted the usability of computational intelligence for forecasting electricity prices.According to them trained ANN model leads to improvement in mean average percentage error (MAPE).Hybrid ANN-BP model has been considered by Fuliang Yin et al. [17] using historical load data for training the neural network.It is observed by them, that ANN with back propagation algorithm improves the training time and convergence towards solution.
Hybrid ANN-GA optimization forecasting models have wide range of applications.In GA search follows the principles of evolution and natural genetics.According to Goldberg [18] GA produces near optimal solutions by following robust search processes.G Aenables optimizing of weights of demand equations.Canyurt et al. [19] studied the dependence of total energy demand of Turkey as a function of economic indicators in linear, quadratic and exponential forms.Ceylon and Ozturk [20], Haldenbilen and Ceylon [21], Assarch et al. [22] analyzed the total energy demand of Iran based on GA.Hybrid ANN optimized with PSO has been successfully applied for load forecasting.
Hybrid ANN optimized with PSO model has been successfully applied for demand forecasting.Bi T et al. [23] and Lu N et al. [24] have used radial basis function neural network for forecasting (RBF).Banda E et al. [25] have presented that time series models lead to large forecasting errors due to their sluggishness to adapt to changing load characteristics.According to their findings ANN-PSO model gives improved results as compared to ANN-BP.Yang S D et al. [26] have incorporated PSO algorithm with an adaptive weight factor to improve the performance.
The remainder of the paper is organized as follows: Section 2 introduces the Electricity sector in Tamil Nadu; Section 3 presents methodology used for research; Section 4 shows the features of ANN-GA-PSO models; Section 5 brings out the results and discussion; Section 6: Conclusions.

The Tamil Nadu Electricity Sector
For more than a decade, the energy sector in Tamil Nadu has witnessed a high growth of industrial activity coupled with exploding domestic electricity demand in the consumer segment.Both of these factors have led to a large deficit in power availability over the last few years.Table 1 brings out the fact that generation capacity has not kept pace with electricity consumption.The immediate solution to the predicament has been the buying of power through short term contract.According to the report of Central Electricity Authority of India (CEA) the electricity deficit of the state in the year 2013 was around 17.5% as compared to 2.8% in the year 2008.Hence there is a dire necessity to forecast the electricity demand by the year 2023 to facilitate the investments in the sector.

Factors Affecting Electricity Demand
Electricity consumption of a state is a function of man affecting factors such as gross state domestic product (GSDP), consumer prices index, energy per capita and income parameters.The following factors reflect their major impacts on electricity demand: (1) GSDP: Even though the linkage between GSDP growth and electricity demand growth are not as strong as it was in the past, it is worth considering the impact on the society of high GDP growth itself since they are linked to each other.A high GSDP growth rate year after year means higher manufacture of products and provision of services at an unprecedented pace leading to higher electricity demand.The electricity demand continues to grow in the state because of high level to continue in a business as usual scenario.

Methodology
In this section, ANN that is optimized by hybrid GA-PSO algorithm in the linear and quadratic forms models the electricity demand.The results of ANN-GA-PSO and A-G-P-Q are compared with ANN with single optimization with GA and PSO algorithms.

Artificial Neural Network
ANN resembles human brain in its origin.It consists of a large number of neurons interconnected to form a complex and non-linearly connected array of parallel network.The most common form of ANN is multilayered perceptron (MLP) which has an input layer with one or more hidden layer connected to one output layer.In present research we have considered a multilayer perceptron (MLP) that has three neurons layers.The first one is the input layer which is in the direct contact with the input data.The middle one is called the hidden layer and it has no contact with outside system.It connects data from the input layer and sends them to the next layer.The last one is the output layer that sends out results.Table 2 gives the network information of ANN about the input layer that is made up of four factors namely, electricity consumption (E.Con), income growth rate, GSDP and Consumer price index.The hidden layer has been used as the activation function.The output layer comprises of one unit representing electricity demand as the dependent variables.The in-sample data is split into two subsets, namely, the training set and the validation set.The training set is then used to train ANN-GA-PSO models until the training error ratio criterion of 0.001 is achieved.The Table 3 shows the sum of squares error, relative error, stopping rule and the training time of the ANN.

Particle Swarm Optimization (PSO)
Particle swarm optimization algorithm was developed by Kennedy and Eberhart in 1995 [27].Over the years PSO has become a popular population based derivative free algorithm.A variant of PSO was developed by Shi and Eberhart [28,29] by inserting a time dependent variable that improves the convergence of the search process.In the previous research paper by the authors [30] following equations have been used for the particle position and the velocity of the particles: where C 1 and C 2 are knowledge factors, R 1 and R 2 are random numbers, g is the location of the leader, p the personal best location, v t is the velocity at iteration "t" and x t is theposition at iteration "t".This equation reveals the particle leader location to each particle.Decreasing the variable enables the slowing down of the speed of the particles around the leader location and provides a balance between exploration and exploitation.PSO finds an optimal point from the random set of points with the help of a fitness function, so that the random points are initialized between the ranges of values of the past two years, which might find the point that matches the straight line formed by the data.This new point is the predicted value for the next year.
GA-PSO hybrid algorithm was first proposed by Bates and Granger [31].According to them linear combination of two forecasting models have a distinct the advantages over individual models.For their application in electricity domain, Nazari et al. [32] proposed a model using two metaheuristic algorithms, namely GA and PSO for forecasting energy demands.They found that the exponential model derived from the PSO model is the best model.Unler [33] proposed PSO based demand forecasting model for Turkey using gross domestic product, population as predictors of energy demand.Younes M et al. [34] provided a solution to the economic dispatch problem using a hybrid method genetic algorithm-particle swarm optimization (GA-PSO).They found that GA-PSO provides flexibility fast convergence, less computational time for non-linear characteristics of power systems.
Araby EE El et al. [35] proposed that a two layered hybrid PSO-SLP (Successive Linear Programming) approach that is suitable for nondifferentiable and discontinuous objective functions.Jarrndal and Hamdan [36] have described a combined approach of artificial neural networks (ANN) with particle-swarm-optimization (PSO) and genetic algorithm optimization (GA) for short and mid-term load forecasting.The model identifies the relationship among load, temperature and humidity using a case study of Sharjah City in United Arab Emirates.They have found that ANN is one of the powerful artificial intelligence techniques for load forecasting which is independent of the human experience [37,38].In the hybrid algorithm PSO is used as a main frame while GA is used as local search that enables PSO to jump out of the local optima.In this way GA-PSO-NN gives a superior generalization capability, low prediction error and optimum network.
When ANN is optimized by a single optimization method such as GA or PSO then it suffers from well-known drawbacks.In the present study, we propose a hybrid algorithm called GA-PSO, which lead to better optimization results.GA-PSO combined optimization algorithm can fully combine merits of single optimization models without their disadvantages.In order to test the accuracy of the models, we have compared the forecast results of ANN-GA-PSO models with other models using single optimization of ANN by GA, single optimization of ANN with PSO, ANN with backward propagation, ARIMA, HOLTS and linear regression.Mean absolute percentage error has been used as an indicator of quality of prediction.It is worth mentioning that, for the sake of comparison among different techniques electricity demand is derived using the same for all the modeling methods.Results point out that ANN optimized by both GA-PSO in quadratic form (A-G-P-Q) gives the best performance followed by ANN-G-P model.Consequently A-G-P-Q model is used to forecast the electricity demand until 2025 based on "as-it-is" scenario and scenario as per the "Vision document" of the state.

ANN-GA-PSO Models
In order to successfully predict Tamil Nadu's electricity demand efficiently and precisely a hybrid GA-PSO based ANN model is proposed here in two-form estimation method.

Two Form Estimation Method
The authors have used the following equations for the GA-PSO optimization: where D is the electricity demand; X i , X j are the factors affecting ith and jth factors affecting electricity energy demand; W o , Y i , k ij and U i are the coefficients and N is the number of demand-affecting factors.PSO searches for the best fitted members that minimize the error.PSO optimizes the weights of socio economic indicators by using both linear and quadratic regression models.Based on these two variations of PSO, models have been named ANN-PSO (Linear) and ANN-PSO (Quadratic) respectively.In PSO-Quadratic, the coefficients of the input variables are calculated as per the Equation (4).For the Quadratic PSO model the quadratic terms are introduced in the following evolution equations: (5) Quadratic PSO algorithm improves the diversity of the swarm leading to higher performance in global optimization.Quadratic PSO projects the input variables for the years 2001 to 2015 based on the data from 1991 to 2000 as input.
In GA, N represents the number of the particles in the population; f i as the fitness value for the individual i.The population size particles are reproduced on the position of the particles using the following equation.
where f max is thelargest fitness value in the generation and p i represents the probability for the selection of the individual i.The crossover and the mutation operations are implemented with p i and p m according to following equations: where X t A and X t B are cross over chromosomes.∝ is a parameter that is constant.

GA-PSO Hybrid Optimization Algorithm
In most of the research papers on the subject, either GA or PSO has been used as single optimization algorithm [39,40].But our research puts forward hybrid GA-PSO algorithm, where GA and PSO are applied serially for providing the best optimizing solution for ANN.PSO optimization is applied to a population of 100 particles and the position and velocity of particle that give the best objective function is arrived at and is designated as 'pbest'.This current best fitness position is compared with the global best.The best global position obtained after PSO optimization is taken as selection value for the GA optimization.In our research paper, the GA further optimizes the best solution thrown up by the PSO.It has been found in our research that hybrid optimization of GA-PSO therefore gives a better solution as compared with single optimization by GA or PSO.As shown in Figure 1 the iterative approach of GA-PSO followed in the study is as follows: Step 1: First, we initialize a population size of 100 and assign positions and velocities of particles.
The number of weights and biases are used to calculate the fitness function for all the particles.Step 2: The best position value achieved by particle p is set as pbest.The pbest with best value is set as gbest and this value is stored.
Step 3: The desired optimization fitness function f (x) is evaluated for each particle.
Step 4: The evaluated fitness value fp of each particle is compared with its pbest value.If fp < pbest then pbest = fp and bestxp = xp, where xp represents the current coordinates of particle p and bestxp represents the coordinates corresponding to particle p's best fitness so far.
Step 5: After objective function value is calculated for new positions of each particle the overall best fitness value of the swarm becomes the gbest value of the swarm.Step 6: Next, the velocity and location of the particle is updated according to Equations ( 1) and ( 2).
The best position is fed into the General Algorithm as selection.
Step 7: The calculation is stopped when the maximum number of iteration reaches 200 or if the convergence occurs before it otherwise Loop to step 3 until convergence.In the present study, the convergence occurs around 50 iterations as shown in Figure 2.
Step 8: The pop size of M particles obtained by GA and M particles are combined to form new pop size particles.
Step 9: Let g en = g en + 1, then step 3 is carried out.
Step 10: The best fitness values and solutions, namely, the position are outputted.

Computational Environment and Data Management
All the GA and PSO techniques have been developed in open source SCILAB environment.For application of ARIMA, HOLTS and Linear models standard econometric toolboxes of IBM SS Software (Version 2, developed by IBM, Armonk, NY, USA) has been used for ANN simulation.It is designed to provide the necessary tools as a part of standard ANN algorithms and relevant analysis.In this research study, the GSDP data is measured in rupees and per capita energy intensity in KWh.The coefficients of predictors as obtained from GA-PSO optimization are depicted in Table 4.The factors are normalized according to the following equation for optimum functioning of ANN.n(x) = (x − xmin)/xmin (7) Table 4 shows the relative values of the independent variables GA-PSO optimization that are used for ANN simulation where E.Con, Income, GSDP, CPI are the input variables.Table 5 indicates the coefficients of Equation (3) obtained by using GA-PSO optimization.Weron R [42] has asserted that MAPE is the most popular evaluation index that works well in load forecasting.Therefore, in order to compare predictive accuracy of the ANN-GA-PSO models, we have used mean absolute percentage error (MAPE) as the evaluation index.The MAPE and forecasting accuracy (τ) have been defined as follows: where A t is the actual value and F t is the forecast value.The total electricity demand of Tamil Nadu from year 2001 to 2015 has been used as a benchmark to test the effectiveness and superiority of the proposed ANN-GA-PSO models.First, ARIMA (1,0,1), HOLTS and linear models have been employed to calculate electricity demand.Secondly the simple optimization of ANN is performed by GA and PSO separately and the results are tabulated under ANN-GA and ANN-PSO respectively.The optimum weight coefficients of GA-PSO optimization are obtained from Equations ( 3) and (4) for ANN-GA-PSO in linear and quadratic forms respectively.For the sake of verifying the validity and superiority of the proposed ANN-GA-PSO models, the comparison is also made with ANN-BP model.

Results
Table 6 and Figure 3 shows results of ANN-G-P and A-G-P-Q models in both linear and quadratic forms along with simple optimization models, ANN-PSO and ANN-GA.Figure 4 compares the errors of linear, Time series models (Holts and ARIMA), ANN-GA, ANN-PSO, ANN-G-P and A-G-P-Q models.Table 7 and Figure 5 compare the MAPE values of different models.It can be seen that MAPE of A-G-P-Q (0.2%) and ANN-G-P (0.3%) are far better than MAPE of single optimized models of ANN-GA (0.42%) and ANN-PSO (0.4%).Table 8 depicts the forecasting accuracy (τ) of different models.It is clear that τ of A-G-P-Q model at 0.78 followed by ANN-G-P at 0.7 are far superior to single optimization models.Figure 6 compares the result of the ANN-G-P (Linear) and A-G-P-Q (Quadratic) model against the actual values of the electricity demand from the year 2001 to 2015.ANN-G-P and A-G-P-Q are in close agreement with the actual values.The forecasts of A-G-P-Q are compared with actual demand on a logarithmic scale in Figure 7.It is seen that the relationship between the two is linear and the slope is 0.99.Thus A-G-P-Q model is best suited for forecasting the electricity demand for the year 2016 to 2025.

Future Estimation
The future estimation of the electricity demand of Tamil Nadu has been evaluated under two scenarios.Scenario 1 (as it is) assumes the energy consumption to grow at the rate of 5%, income at the rate of 12%, GSDP at 11% and CPI at 2%.Scenario 2 considers the VISION Document 2023 [43] goals of the state as expected growth rate of energy consumption as 8%, income growth as 15%, GSDP as 12% and CPI at 3%.Table 9 shows the tabulated results of the forecasted electricity demand for scenario 1 and scenario 2 using A-G-P-Q model.Figure 8 shows the forecasted electricity demand as per scenario 1 and scenario 2. The projected electricity demand as per scenario 2 are on the higher side throughout except for the year 2020.The electricity requirement for the year 2025 is 84 GWh as compared to 87.8 GWh as per scenario 1 and scenario 2 respectively.The state of Tamil Nadu will have to find resources for fulfilling the demand of 87.8 GWh if it wants to achieve the goals set up by the Vision 2023 document.

Relationship between GSDP and Electricity Demand
According to Kostyannikova D [44], the causality and co-integration relationship between GSDP and electricity demand are not uniform across countries due to difference in policies and energy structure.Our present study shows that electricity demand and GSDP are co-integrated in the case of Tamil Nadu.As shown in Figure 9 one percent increase in total energy consumption leads to an increase of 0.86 in GDP while one percent increase in GSDP will raise total energy consumption by 0.79 percent.
Our research shows that in case of Tamil Nadu, causality exists between GSDP and electricity demand.Hence it will be possible to increase the GSDP by investing in bridging the electricity demand gap.

Conclusions
This study has proposed a novel algorithm based on PSO and GA for optimizing ANNs in linear and quadratic forms for forecasting of electricity demand.ANN has been optimized by the hybrid optimizing algorithm of PSO and GA in linear and quadratic forms.Single optimized ANN (ANN-GA, ANN-PSO) have been compared with hybrid optimized ANN's (ANN-GA-PSO, A-G-P-Q).ANN-GA-PSO models in linear and quadratic forms have demonstrated 28% and 48% improvement over ANN-GA model and 25% and 43% improvement over ANN-PSO model.ANN-GA-PSO models can solve the problem of over fitting and falling in local minimum in data set ANN-GA-PSO model shave been used to explore the relationship between electricity demand and GSDP of Tamil Nadu state which is seen as co-integrated.ANN-GA-PSO models can be used for resource planning and for bridging the energy gap in the state to achieve the goals set out in the Vision document of the state.

( 2 ) 3 )
Electricity consumption per capita (E.Con) has increased from 510 kWh in year 2000-2001 to 1065 kWh in 2011-2012, that is more than 100% increase.Hence per capita consumption has been taken as an independent factor.(Income growth rate (per capita): The vision 2023 document of the state of Tamil Nadu aims at doubling the per capita income by 2023.It is also seen that any increase in family income leads to spurt in consumption.(4) Consumer Price Index (CPI): Prices have an indirect impact on the electricity demand by affecting the purchase of luxury goods such as air conditioners, washing machines etc.

Figure 4 .
Figure 4. Error comparison of various models.

Figure 8 .
Figure 8. Forecasts as per scenario 1 and scenario 2 using A-G-P-Q model.

Table 5 .
[41]ficients of GA-PSO Linear.Root Mean Square Error (RMSE) or Mean Absolute Error (MAE) is commonly used as a measure of forecasting performance.However, Fatai and Armstrong[41]have negated RMSE or MAE, as both are scale dependent and RMSE is affected by outliers that are common in electricity forecasting.

Table 6 .
Performance of different models.