Fusion of Improved Sparrow Search Algorithm and Long Short-Term Memory Neural Network Application in Load Forecasting

Abstract

Load forecasting (LF) is essential in enabling modern power systems’ safety and economical transportation and energy management systems. The dynamic balance between power generation and load in the optimization of power systems is receiving increasing attention. The intellectual development of information in the power industry and the data acquisition system of the smart grid provides a vast data source for pessimistic load forecasting, and it is of great significance in mining the information behind power data. An accurate short-term load forecasting can guarantee a system’s safe and reliable operation, improve the utilization rate of power generation, and avoid the waste of power resources. In this paper, the load forecasting model by applying a fusion of Improved Sparrow Search Algorithm and Long Short-Term Memory Neural Network (ILSTM-NN), and then establish short-term load forecasting using this novel model. Sparrow Search Algorithm is a novel swarm intelligence optimization algorithm that simulates sparrow foraging predatory behavior. It is used to optimize the parameters (such as weight, bias, etc.) of the ILSTM-NN. The results of the actual examples are used to prove the accuracy of load forecasting. It can improve (decrease) the MAPE by about 20% to 50% and RMSE by about 44.1% to 52.1%. Its ability to improve load forecasting error values is tremendous, so it is very suitable for promoting a domestic power system.

1. Introduction

Power systems are vital to people’s lives, business activities, industrial development, and scientific research, and can contribute significantly to the development of socio-economic science. The fundamental energy source of an electric power system is primary energy, which, by converting it into electricity, supplies the electricity that people need to carry out their lives, consisting mainly of power generation, transmission, substation, distribution, and electricity use. Therefore, to ensure that users have a safe and reliable power supply, power systems are a solid backing for society and people to achieve efficient energy use. The power load is objective; when it is too light or too heavy, it is not conducive to the power system’s safety, stability, and economic development, so the power system load forecast has many necessities and significance. With the continuous improvement of Taiwan’s installed power generation capacity, the trend of power load changes is becoming more complex, bringing significant challenges to the planning and operation of the power system. Considering the need for a power system to plan future power consumption and master the power consumption situation, it is of great reference value to optimize the power supply process from timeliness, accuracy, and reliability. For short-term load forecasting, the improvement of accuracy and forecasting speed is not only beneficial to the reliable power supply and stable operation of the power system, but also to the control of the power quality, and thus to the power supply system on the social and economic levels. Therefore, it is of great research value to study the improvement of the forecasting model of short-term load forecasting and thus improve the reliability of its forecasting results. Short-term load forecasting is an important research area for the operation and optimization of the power system. However, there are factors of randomness and uncertainty of power load. Thus, accurate forecasting is not easy.

Countless domestic and foreign scholars in this field have carried out substantial research on the earliest artificial forecasting methods and traditional statistical forecasting methods, up to today’s machine learning forecasting methods and combination model forecasting methods. Researchers put forward many effective algorithmic schemes to obtain appropriate research results in their respective eras. These methods include the following: (1) Traditional statistical forecasting methods. The traditional statistical forecasting method is mainly composed of the Regression Analysis method [1,2,3], the Time Series method [4], the Kalman Filter method [5], the Least Square Method [6], neural networks (NNs) [7,8], etc. The basic idea is to build a mathematical model—the optimal combination of multiple influencers of the load or historical data from the last moments of the load. The advantage is that the model has a fast and straightforward forecasting speed, but it is poor for nonlinear data forecasting performance. Once the historical data are missing or have a significant error, it is impossible to construct an ideal mathematical model, affecting forecasting accuracy. With the continuous progress of the power system, the data of power load are growing larger, and the characteristics of factors affecting the change of load are increasing and becoming more complicated. The limitations of the traditional statistical forecasting method are further highlighted, and the scope of application is narrowed step by step. (2) Machine learning forecasting methods. The basic idea of machine learning forecasting and computer technology is based on probability theory, statistics, approximation theory, convex analysis, algorithmic complexity theory, and other disciplines. The calculator’s mighty computing power can mine potential information from large amounts of data, learn from experience, improve itself, and automatically build generalized models for forecasting. The learning strategy can also be subdivided into machine learning that directly uses mathematical methods, including Support Vector Machine (SVM) [9,10], Random Forest (RF) [11], etc. Typical representatives and intelligent algorithms that simulate the learning process of neurons in the human brain include BP Neural Network (BPNN) [12], Machine Learning (ML) [13,14,15], Deep Neural Network (DNN) [16,17], Convolutional Neural Network (CNN) [18,19,20,21], Recurrent Neural Network (RNN) [22,23,24], and Long Short-Term Memory Neural Network (LSTM) [25,26,27,28,29]. Support vector regression (SVR) [30,31,32,33] has the advantages of structural stability, generalization ability, optimal global solution, etc., but its ability to handle significant sample data performance is poor and not sensitive enough to timing data. (3) The combined model forecasting method. Although machine learning forecasting can make up for many problems of traditional statistical forecasting methods, each model has an inherent defect. Selecting historical load characteristics and establishing models can break the continuity and intrinsic laws of the load. With the development of deep learning, the circular neural network model, represented by the Gated Recurrent Unit Network (GRU) [34,35,36], can dig up the potential information in the timing data on its own and describe the change law of the timing data comprehensively.

However, there is also significant sample demand, susceptibility to overfitting, poor generalization ability, etc. Based on the uncertainty and multi-scheme of complex forecasting, the combined model combines multiple single models to obtain more accurate forecasting results and a more robust forecasting model. Such combination models are commonly able to obtain more significant and superior generalization and forecasting performance than a single model. Dash et al., for example, propose a combination model that combines neural networks with Fuzzy Expert Systems (FES) for short-term load forecasting. After the historical load data and weather data are blurred by the membership function, as the neural network input, the network output is also blurred by the membership function, and then the FES is entered. An adaptive fuzzy correction scheme obtains the final load forecasting.

Another example is Che et al., who proposed a support vector regression model based on different “kernels”, using different kernel functions to establish a single-kernel support vector regression model and then combined them according to the inverse ratio of errors. Jihoon et al. sought to overcome the single layer of a deep neural network optimal, super parametric, hidden number, which made it complicated to determine the problem based on the training set. They constructed four different four-layer deep neural network forecasting models, validated and selected super-optimal parameters, and finally, using the primary component regression method based on sliding windows, built a constructive load forecasting model.

In contrast, in this paper, more effective solutions are proposed for some of the shortcomings of these new methods. We propose to establish an electric load forecasting model by applying fusion of the Improved Sparrow Search Algorithm (ISSA) [37,38,39] and Long Short-Term Memory Neural Network (ILSTM-NN). The ISSA is proposed to optimize the parameters of the LSTM-NN as a new method of short-term load forecasting. Then, the nonlinear approximation capability of the ILSTM-NN method itself is used to improve the accuracy of the solution load forecasting. At the same time, we can obtain more accurate load forecasting when dealing with the load under the influence of various factors (such as temperature, humidity, rainfall, holidays, etc.) and consider the individual factors.

2. How Power Load Forecasting Works

Load forecasting is based on meteorological conditions, date patterns, social impact, and system operation status, and a mathematical model for dealing with the past and future loads of electric power is established to determine the power load data at a particular time in the future under the condition of meeting the requirements of specific forecasting accuracy. Set the influence factor of the power load $\left\{l_1,l_2,\dots ,l_p\right\}$, where p represents the number of influence factors and y represents the power load observation value, using a particular model method to describe the nonlinear relationship between the power load of the influence factor, as in Equation (1):

$$y=f(l_1,l_2,\dots ,l_p)$$

(1)

3. Short-Term Load Forecasting Is Described Using the ILSTM-NN

Because the power load is a complete result influenced by many factors, showing a high nonlinear variation law and a limited sample for forecasting, it is difficult for traditional linear methods to forecast accurately based on extensive sample data. Therefore, the ILSTM-NN method is proposed here because it has infinite nonlinear approximation capability and is suitable for power load forecasting. Using an ILSTM-NN to forecast power system load, the first step is to determine the model input, the historical power load, as the essential input, because it is the most crucial factor of the model’s forecasting accuracy improvement. We need to consider the conditions that may affect the change in the load. The correlation between the power load data and the influence factor data is analyzed so that the correlation is extensively filtered out, which helps to improve the model’s forecasting accuracy, and the influence factor data are used as the input of the model. If the input data are not carefully processed and entered directly into the model, the training effect of the model will be poor, so the data input model needs to preprocess the data, mainly to supplement the lack of data, in addition to reducing the data scale of the input data. The preprocessed data can be entered into the model for training, and the trained model outputs forecast data to observe the forecasting effect of the model by comparing the forecasting data with the actual data. Suppose forecasting results do not meet the requirements. In that case, it is necessary to adjust the model parameters to retrain the model and have the forecasting results be compared again until they meet the requirements. The flow chart of ILSTM-NN for power system load forecasting is shown in Figure 1.

4. Recurrent Neural Network (RNN)

Load forecasting essentially finds a mapping relationship in historical data and reuses it to forecast future power load data. The neural network model has strong memory neural networkability, robustness, and nonlinear learning ability, which can solve power system load forecasting well. LSTM-NN is an improved circulatory neural network algorithm and RNN works in a loop; that is, the output of the last neuron, together with other variables at the next point in time, is re-delivered to the current neuron as an input to the neuron, the data in the loop from the current state to the next state, so the RNN can be expanded to a chained neural network. Figure 2 shows the expanded schematic of the RNN.

In Figure 2, the input for the neural network unit is Xt, and the output is Ht. RNN is effective at learning and forecasting historical information when the training data are used at short intervals. RNN can still learn historical information and make accurate forecasts. As the time interval of training data increases, the learning ability will weaken. The model will increase with the interval, forget the law of the previous data, the long-term dependence of poor learning ability, and thus, the phenomenon of gradient decline. Power load forecasting needs to learn the long-term load data, find the mapping relationship, and perform forecasting, and the RNN deals with the load forecasting problem; overall, it is challenging to achieve the desired effect.

5. Long Short-Term Memory Neural Network

When an RNN is dealing with a problem with long-term dependency, network-generated gradients disappear, and gradient explosion occurs because back-propagation causes bias to very small or very large values when updating parameters. The birth of LSTM is to better deal with such problems. An LSTM network over the course of training can spontaneously remember the long-term information of data and its relevance in various tasks to achieve suitable predictive results, so it is widely used in machine translation, time series forecasting, and other fields. It is also more helpful than other statistical methods. However, the LSTM-NN model is more complex than the neuron structure of the RNN model, and information is transmitted one by one in the cells of the network (as shown in Figure 3).

To avoid gradient decline, as in RNN, LSTM-NN uses the “gate” structure to enhance information transmission and communication between cells, including the input gate, the output gate, and the forgetting gate, which learns and forecasts the load data of the power system by controlling the input, output, and historical dependence of neurons. The LSTM-NN operation is as follows:

$$f_t=\sigma (W_{\mathrm{f}}\times [h_t-1,x_t]+b_{\mathrm{f}})$$

(2)

$$i_t=\sigma (W_{\mathrm{i}}\times [h_t-1,x_t]+b_{\mathrm{i}})$$

(3)

$${\tilde{c}}_t=\tanh (W_c\times [h_t-1,x_t]+b_c)$$

(4)

$$c_t=f_t\times c_t-1+i_t\times {\tilde{c}}_t$$

(5)

$$o_t=\sigma (W_o\times [h_t-1,x_t]+b_o)$$

(6)

$$h_t=o_t\times \tanh (c_t)$$

(7)

Equation (2) controls the “forgotten” information of the current neuron from the previous neuron, implemented by the Sigmoid layer known as the “forgotten gate”, $\sigma$, which is an activate function known as a Sigmoid function. $W_{\mathrm{f}},b_{\mathrm{f}}$ are the weights and thresholds of the forgotten gate, respectively. The forgotten gate works by reading the last neuron’s output, $h_{t-1}$, and the current neuron’s input, $X_t$, entering a value between 0–1 and multiplying with the previous cell state $c_{t-1}$. If the output is 1 multiplied by $c_{t-1}$, representing the previous cell state information, this is a “memory neural network”. If the output is 0 and multiplied by $c_{t-1}$ to represent the previous cell state information, this is completely “forgetting”. In this way, the long-term memory of the network model is guaranteed. Equations (3) and (4) together control the input of cells. Equation (3) is implemented by the Sigmoid layer called the “input gate”. After the input gate reads the output $h_{t-1}$ of the previous neuron and the current neuron input $X_t$, it outputs a quantity of 0–1, $i_t$. In Equation (4), a new candidate vector ${\tilde{c}}_t$ created by a layer tanh can obtain the current cell state $c_t$ of Equation (5) through $c_{t-1}$, $f_i$, $i_t$. $W_{\mathrm{i}},W_c$ are the weight matrix of the input gate and the candidate value, respectively; $b_{\mathrm{i}},b_{\mathrm{c}}$ are the corresponding bias values.

After calculating the information retention of the previous neuron and the input information of the current neuron, the final output is controlled by the Sigmoid layer called the “output gate”. Multiply the current cell state $c_t$ treated by the tanh layer with the Sigmoid $o_t$ (as shown in Equation (6)) obtained by the output gate, and the final output $h_t$ of the current neuron in the resulting Equation (7). $W\mathrm{o},b\mathrm{o}$ are the output gate’s weight and bias values, respectively.

6. Optimization of Load Forecasting Model with Improved Sparrow Search Algorithm (ISSA, Also Named Chaos Sparrow Search Algorithm, CSSA)

6.1. Optimization of LSTM-NN Parameters

The key to studying load forecasting of power systems using LSTM-NN lies in training models. Furthermore, the ultimate purpose of model training is to find a set of optimal parameters (that is, the number of hiding neurons, weight values, bias values, and the number of iterations) so that the forecasting result can be the most accurate. However, the diversity, uncertainty, and correlation of model parameters make determining model parameters extremely complicated. The optimization problem is to find a set of parameter solutions to the problem under certain constraint conditions and have this set of solutions meet the optimality of the problem measurement so that the performance index of the whole system can reach the optimal condition. We can use the following two styles to describe the optimal problem:

$$\begin{gathered} min/max\left\{F(x)=(f_1(x),f_2(x),\dots ,f_n(x))\right\} \\ s.t. x\in S=\left\{x\left|g_i(x)\le 0,i=1,2,\dots ,n\right.\right\} \end{gathered}$$

(8)

In Equation (8), F(x) is the optimized target function, $g_i$ is the constraint function, and x is the decision variable. For the multi-objective optimization problem, there may be a contradiction between the target functions it contains, so the process of solving needs to balance this contradiction so that the n-objective functions can reach the optimal value simultaneously.

6.2. CSSA Optimizes LSTM-NN

6.2.1. Model Design

The LSTM-NN model is used for power system load forecasting. However, the key to model training lies in selecting model parameters, and selecting model parameters by experience alone will significantly increase the training difficulty of the model and lead to the model’s training not having the best forecasting ability. Using a CSSA to optimize the parameters in the LSTM-NN model can significantly reduce the artificial intervention in the model’s training process, and finally, the resulting model has the best forecasting effect.

6.2.2. The Basic Principles of the CSSA

The Chaos Sparrow Search Algorithm is a new swarm intelligence optimization algorithm that simulates sparrow foraging behavior. Because it is not limited by whether the target function can be divided, directed, continued, and other characteristics, and has the advantages of better stability and high efficiency. The algorithm first randomly initializes a set of solutions and then updates them over generations until the best solution is found. Compared with other intelligent optimization algorithms, CSSA uses simple concepts, transparent processes, and fewer parameters that need to be adjusted, coupled with the traversal, randomness, and regularity of chaotic algorithms, making them search for—and not fall into the trap of—locally optimal solutions. Therefore, this method is easier to achieve when searching for the best solution, and many scholars have studied it.

6.2.3. CSSA Designs the Appropriate Rules

In the Chaos Sparrow Search Algorithm, we divide sparrows into two categories, the discoverers and the adders, who are responsible for finding food in the clans and providing foraging areas and directions for the entire sparrow family. At the same time, the adders use the discoverers to obtain food, and birds are usually flexible in using these behavioral strategies, which can switch between the individual behaviors of the discoverer and the adders. Based on the descriptions of the sparrows, we can build a mathematical model of the Sparrow Search Algorithm. In order to make the algorithm more concise, we will idealize the following behavior of the Sparrow Search Algorithm and establish the corresponding rules. The main rules are shown below.

• Discoverers usually have high energy reserves. They are responsible for searching for food-rich areas throughout the species, providing foraging areas and directions for all adders, and in modeling, the level of energy reserves depends on the individual sparrow and the corresponding adaptability values.
• Once the sparrow finds the predator, the individual sends out an alarm signal, and when the alarm value is greater than the safe value, the finder takes the adders to another safe area foraging.
• The identity of the discoverer and the adders is dynamic, and each sparrow can be the discoverer as long as a better food source can be found. However, the proportion of the discoverers and the adders in the entire population is constant. When one sparrow becomes a discoverer, another sparrow becomes the adder.
• The lower the adders' energy, the worse their foraging position throughout the species, and some hungry adders are more likely to fly elsewhere for food and more energy.
• During foraging, adders can always search for the finder that provides the best food, or forage around the finder, while some adders may constantly monitor the finder to compete for food resources to increase their predation rate.
• When aware of the danger and what will happen, the sparrow at the edge of the family moves quickly towards a safe area to find a better position. The sparrow in the middle of the family will only move randomly, close to the other sparrows.

6.3. The Design and Steps of the CSSA

6.3.1. The Initial Group Is Generated

We can use logistic mapping to generate r chaotic variables:

$$x_{z+1}^s=\mu \cdot x_z^s(1-x_z^s)s=1,2,\dots ,r$$

(9)

where $\mu =4$. Let z = 0 in Equation (9). Given different initial values for r chaotic variables, using (9) produces r chaos variables $x_1^s$(s = 1, 2, …, r). The first sparrow in this r chaotic variable is used to initialize the population, and the z is 1, 2, …, v, followed by the above method to produce another v−1 sparrow, which forms the initial population.

6.3.2. Design Adaptation Functions

Since the goal of LSTM-NN parameter optimization is to ensure that the forecasting accuracy of the forecasting model is as high as possible, the following two quantities are used to evaluate the load forecast error value:

$$MAPE=\frac{1}{n_d}{\displaystyle \sum _{pr=1}^{n_d}\frac{\left|y_t-{\overline{y}}_t\right|}{y_t}}\times 100\%$$

(10)

$$RMSE=\sqrt{\frac{1}{n_d}{\displaystyle \sum _{pr=1}^{n_d}{\left(y_t-{\overline{y}}_t\right)}^2}}$$

(11)

In Equations (10) and (11), $y_t$ represents the actual value, ${\overline{y}}_t$ represents the forecasting value, MAPE represents mean absolute percentage error of the forecasting, RMSE represents root mean square error of the forecasting, and $n_d$ represents the training sample pair. These two values effectively evaluate forecasting power, so these two criteria are chosen to evaluate the model’s merits in the test.

6.3.3. Sparrow Search Acting Algorithm

Each sparrow (also each solution) can be expressed as:

$$\begin{gathered} X_i=[x_{i,j},\dots ,x_{i,D}] \\ i=1,2,\dots ,n, j=1,2,\dots ,D \end{gathered}$$

(12)

The adaptability values of all sparrows can be expressed as follows (13):

$$F(X)=\left(\begin{array}{l}{f}_1([x_{1,1} x_{1,2} \dots x_{1,D}])\\ f^2([x_{2,1}{x}_{2,2} \dots x_{2,D}])\\ ⋮\\ ⋮\\ f_n([x_{2,1}{x}_{2,2} \dots x_{2,D}])\end{array}\right)$$

(13)

where D denotes the dimensionality of the problem variable to be optimized.

The CSSA algorithm is a new type of swarm intelligent optimization algorithm proposed by sparrow foraging behavior and anti-predation behavior. The bionic principle is as follows: the sparrow foraging process can be abstracted into the discoverers and adders model. Adding reconnaissance and early warning mechanisms, the discoverer adapts highly, searches widely, and guides the swarm search and forage. The adders, for better adaptability, follow the discoverer’s foraging, and improve their own feeding rate by monitoring the discoverer for food competition or foraging around it, and when the entire swarm is threatened by predators or aware of the danger, adders immediately act as anti-predators, simulating the sparrow foraging process in the CSSA to obtain an optimized solution to the problem. Assuming that there are n sparrows in a D dimensional search space, the ith will only be the position of the sparrow in the $X_i=[x_{i,j},\dots ,x_{i,d},\dots ,x_{i,D}]$ $i=1,2,\dots ,n$, $j=1,2,\dots ,D$, represents the ith sparrow in the d dimensional position. In optimizing the LSTM-NN parameters, the first set of solutions is represented as i = 1, while D represents the total number of parameters optimized for the LSTM-NN.

Discoverers generally account for 10% to 20% of the swarm, and the location update equation is as follows:

$$\begin{gathered} X_{i,j}^{t+1}=\left\{\begin{array}{lll}{x}_{i,j}^t\cdot e^{-(\frac{i}{\alpha \cdot {\mathit{iter}}_{\max }})}& if& {\mathrm{A}}_{\mathrm{L}}<\mathrm{SA}\\ x_{i,j}^t+\mathrm{RP}\cdot {\mathrm{MT}}_{\mathrm{X}}& if& {\mathrm{A}}_{\mathrm{L}}\ge \mathrm{SA}\end{array}\right. \\ i=1,2,\dots ,n j=1,2,\dots ,D \\ t: \mathrm{Number} \mathrm{of} \mathrm{generations} \end{gathered}$$

(14)

In Equation (14), t represents the current number of iterations, ${\mathit{iter}}_{\max }$ is the most significant number of iterations, $\alpha$ is the uniform random number between (0,1], and RP is the random number of quasi-normal distribution. MTx is a matrix of 1× d dimension; all its elements are “1”. ${\mathrm{A}}_{\mathrm{L}}\in$ [0,1] and $\mathrm{SA}\in [0.5,1]$ represent the warning value and safety value, respectively. When ${\mathrm{A}}_{\mathrm{L}}<\mathrm{SA}$, the population does not find the presence of predators or other dangers. The finder can widely assess the environment’s security and guide the swarm to achieve greater adaptability. When ${\mathrm{A}}_{\mathrm{L}}\ge \mathrm{SA}$, scouting sparrows spot predators and immediately release red flags, and the swarm immediately performs anti-predator behavior to adjust the search strategy, quickly moving closer to a safe area. In addition to the discoverer, the remaining sparrows are added and updated according to Equation (15):

$$X_{i,j}^{t+1}=\left\{\begin{array}{lll}Q\cdot e^{-(\frac{{x_{worst}}_j^t-x_{i,j}^t}{i^2})}& if& i>\frac{n}{2}\\ {x_{best}}_j^{t+1}+\frac{1}{D}{\displaystyle \sum _{j=1}^D}(rand\left\{-1,1\right\}\cdot \left|x_{i,j}^t-{x_{best}}_j^{t+1}\right|)& if& i\le \frac{n}{2}\end{array}\right.$$

(15)

In Equation (15), where ${x_{worst}}_j^t$ means in the $t$ generation population, the sparrow is in the worst position in the j-dimension, ${x_{best}}_j^{t+1}$, which means that in the $t+1$ generation population, the sparrow is at in best position in the j-dimension. $i>\frac{n}{2}$ means that the ith adders have no food, are hungry, are less adaptable, and need to fly elsewhere for food to gain higher energy, and when $i\le \frac{n}{2}$, this means the ith adders can randomly find a location near the optimal location $x_{best}$ for foraging.

Reconnaissance warning sparrows generally account for 10–20% of the total location, updated as follows (16):

$$X_{i,j}^{t+1}=\left\{\begin{array}{lll}{x_{best}}_j^t+\beta (x_{i,j}^t-{x_{best}}_j^t)& if& f_i\ne f_{best}\\ x_{i,j}^t+K\left(\frac{x_{i,j}^t-{x_{worst}}_j^t}{\left|f_i-f_{worst}\right|+\epsilon }\right)& if& f_i=f_{best}\end{array}\right.$$

(16)

In Equation (16), $\beta$ is represented as a step length control parameter, which is a normal distribution random number with a mean of 0 and a variance of 1. K is a random number between [−1,1], indicating the direction of the sparrow’s movement, and the step control parameter, $\epsilon$, is a very small constant, to avoid a denominator of 0. $f_i$ represents the fitness value of the ith sparrow. $f_{best}$ and $f_{worst}$ represents the optimal and worst fitness of the current sparrow swarm, respectively. When $f_i\ne f_{best}$, the sparrow is on the verge of the swarm, where it can be attacked easily by predators. When $f_i=f_{best}$, the sparrow is in the middle of the swarm, aware of the predator’s threat, and to avoid the predator attack, it will approach other sparrows to adjust the search strategy.

6.3.4. CSSA’s Training Process for LSTM-NN

The weight and bias value are initially taken between 0 and 1, while the number of neurons in the hidden layer and the number of iterations are real values. In the beginning, we can use a larger number. While CSSA is training, it adjusts the size of these four values based on the size of the adaptive values obtained after calculation. Such a continuous cycle operation will finally obtain an optimal value. These values will be recorded as the best values of this training. Then, we enter the following pieces of information and repeat the above training. This time will also obtain another value based on the size of the riding adaptation value, comparing which value will make the load prediction error value low. We record these data and discard the poorer value so that the training data are entered in sequence, and finally obtain the final best solution.

7. Use ILSTM-NN for Short-Term Load Forecasting Models

After the initial LSTM-NN accepts the optimal solution of the network parameters calculated by CSSA, it returns for actual load forecasting. Each calculation output represents the load forecasting value for the current hour, and there are four essential items of the training data entered at the time of forecasting: (1) The historical data of the load, i.e., the collection of load data used in the past, are used as historical data for training (including (i) the load data for the same hour as today, two weeks before the forecast date (e.g., Tuesday); (ii) the load data for the same hour as today, one week before the forecast date; and (iii) the load data for this hour of the forecast day). (2) The temperature information, the maximum and minimum temperature of the day before the load forecasting, and the temperature forecast for the hour in which the load forecasting will be made. (3) Rainy day data collection for the current day. (4) Holiday data collection, whether the holiday constitutes vital judgment data. We mainly perform offline training, and then perform online load forecasting to save real load forecasting time.

8. Simulation and Results Using Load Forecasting for a Region as an Example

8.1. Load Forecasting for Different Dates

Usually, we perform the training work offline first and then go online to perform the actual forecasting work after the training is completed. The software used in this paper is Python 3.8. The training data used in this example are all data for the period from 12 March 2019 to 11 March 2020. The test data are from 10 May 2020 to 9 June 2020, and the actual forecasting for the load is from 1 January 2020 to 31 December 2020. CSSA (at the beginning, CSSA n = 30, D = 75) is used to search for the optimal values for network parameters for LSTM-NN, while LSTM-NN performs load forecasting of LSTM-NN after receiving the solution parameter values sent from CSSA training. In Figure 4, Figure 5 and Figure 6, the load forecasting graph for the actual date is used. In each figure, the solid line represents the actual load, and the dotted line represents the forecasted load. Shown in Figure 4 is the load forecasting for Monday, 7 September with a minimum load (22,571 MW) at about 4:00 a.m. and a maximum load (27,615 MW) at about 4:00 p.m. Shown in Figure 5 is the load forecasting for Wednesday, 16 September with a minimum load (25,470 MW) at about 4:00 a.m. and a maximum load (36,959 MW) at about 3:00 p.m. Moreover, shown in Figure 6 is a week-long holiday; on Saturday, 17 October, the curve in the figure shows that its load value is smaller than the typical working day, and the lowest load occurs at 4:00 a.m. (20,671 MW), while the maximum load (26,615 MW) is around 4:00 p.m.

Furthermore, the load graph shown in Figure 7 is a load forecasting for the typical spring week of 2020 (Monday 11 May 2020 to Sunday 17 May 2020). The peak load of the typical weekly load forecasting is about 34,200 MW, and the point at which the daily peak load occurs is slightly different. Figure 8 is load forecasting for the typical summer week of 2020 (Monday 20 July 2020 to Sunday 26 July 2020). The peak load of the typical weekly load forecasting is about 37,235 MW. Finally, in Figure 9, the load graph is a load forecasting for the typical autumn week of 2020 (Monday 21 September 2020 to Sunday 27 September 2020). The peak load of the typical weekly load forecasting is about 27,350 MW.

8.2. Load Forecasting of Different Methods of Different Dates then Obtaining MAPE and Max MAPE

In this paper, the MAPE and Max. MAPE obtained from different day patterns can be predicted using different methods:

• ANN: Artificial Neural Network method.
• EP-ANN: With ANN as the main body of the forecasting model, and uses Evolutionary Programming (EP) to train its parameters.
• GA-LSTM: With LSTM as the main body of the forecasting model, and uses Genetic Algorithm (GA) to train its parameters.
• PSO-LSTM: With LSTM as the main body of the forecasting model, and uses Particle Swarm Optimization (PSO) to train its parameters.
• ILSTM-NN: The method used in this paper.

For all five methods, the total unit of input layers in this paper is 10. As shown in

Table 1. A comparison of the load forecasting error values ($MAPE$, Max. $MAPE$, and RMSE) for different dates by different methods from 5 July to 20 October 2020.

Day Type	ANN			EP-ANN			GA-LSTM			PSO-LSTM			ILSTM-NN
	$\mathit{M}\mathit{A}\mathit{P}\mathit{E}$	$\mathbf{Max}. \mathit{M}\mathit{A}\mathit{P}\mathit{E}$	RMSE (MW)	$\mathit{M}\mathit{A}\mathit{P}\mathit{E}$	$\mathbf{Max}. \mathit{M}\mathit{A}\mathit{P}\mathit{E}$	RMSE (MW)	$\mathit{M}\mathit{A}\mathit{P}\mathit{E}$	$\mathbf{Max}. \mathit{M}\mathit{A}\mathit{P}\mathit{E}$	RMSE (MW)	$\mathit{M}\mathit{A}\mathit{P}\mathit{E}$	$\mathbf{Max}. \mathit{M}\mathit{A}\mathit{P}\mathit{E}$	RMSE (MW)	$\mathit{M}\mathit{A}\mathit{P}\mathit{E}$	$\mathbf{Max}. \mathit{M}\mathit{A}\mathit{P}\mathit{E}$	RMSE (MW)
Working days	1.93	2.57	447	1.89	2.31	438	1.86	2.05	431	1.72	2.03	399	1.26	1.99	292
Non-working days	1.95	2.68	452	1.92	2.35	445	1.92	2.18	445	1.82	2.12	422	1.28	2.01	297
Public holidays	1.95	2.75	452	1.94	2.41	450	1.97	2.23	456	1.92	2.15	445	1.30	2.02	301
Rainy days	2.01	2.77	466	1.98	2.51	459	1.98	2.27	459	1.97	2.18	456	1.32	2.02	306
Average	1.96	2.71	455	1.93	2.40	447	1.93	2.18	447	1.86	2.12	431	1.29	2.01	299

, in the case of working days, the $MAPE$ value calculated using the ILSTM-NN method is different from ANN, EP-ANN, GA-LSTM, and PSO-LSTM; the calculated $MAPE$ values were reduced (improved) by 55%, 43%, 36%, and 21%, respectively. In the case of non-working days, the values calculated using the ILSTM-NN method are lower than ANN, EP-ANN, GA-LSTM, and PSO-LSTM; the values calculated from the other four methods decreased (improved) by 33%, 31.4%, 20.4%, and 9.2%, respectively. For example, in the case of public holidays, the $MAPE$ value calculated using the ILSTM-NN method is 33.3%, 27.4%, 22.2%, and 16.1% lower (improved) than the values calculated by ANN, EP-ANN, GA-LSTM, and PSO-LSTM, respectively. In the case of rainy days, the MAPE value calculated using the ILSTM-NN method is lower than ANN, EP-ANN, GA-LSTM, and PSO-LSTM by 31%, 29.2%, 24.3%, and 16.5%, respectively. Finally, if calculated in terms of the average $MAPE$ for different day patterns, the average $MAPE$ calculated using the ILSTM-NN method, compared to ANN, EP-ANN, GA-LSTM, and PSO-LSTM, improved by 49%, 41%, 30%, and 22%, respectively. In terms of ten average RMSE, the average RMSE calculated using the ILSTM-NN method, compared to ANN, EP-ANN, GA-LSTM, and PSO-LSTM, improved by 52.1%, 49.5%, 49.5%, and 44.1%, respectively.

Regarding the monthly differences for $MAPE$ and Max. $MAPE$, the averages obtained using different methods for the whole year from January 2020 to December 2020 are shown in

Table 2. The average $MAPE$ and Max. $MAPE$ of each month was obtained using different methods for different months, from 1 January 2020 to 20 December 2020.

Month	ANN		EP-ANN		GA-LSTM		PSO-LSTM		ILSTM-NN
	$\mathit{M}\mathit{A}\mathit{P}\mathit{E}$	$\mathbf{Max}. \mathit{M}\mathit{A}\mathit{P}\mathit{E}$	$\mathit{M}\mathit{A}\mathit{P}\mathit{E}$	$\mathbf{Max}. \mathit{M}\mathit{A}\mathit{P}\mathit{E}$	$\mathit{M}\mathit{A}\mathit{P}\mathit{E}$	$\mathbf{Max}. \mathit{M}\mathit{A}\mathit{P}\mathit{E}$	$\mathit{M}\mathit{A}\mathit{P}\mathit{E}$	$\mathbf{Max}. \mathit{M}\mathit{A}\mathit{P}\mathit{E}$	$\mathit{M}\mathit{A}\mathit{P}\mathit{E}$	$\mathbf{Max}. \mathit{M}\mathit{A}\mathit{P}\mathit{E}$
January	2.02	2.92	1.84	2.11	1.81	2.09	1.72	2.08	1.19	2.02
February	1.99	2.87	1.92	2.25	1.86	2.57	1.86	2.12	1.22	2.06
March	2.03	2.52	1.91	2.18	1.88	2.12	1.52	2.02	1.21	2.03
April	1.92	2.76	1.98	2.32	1.98	2.35	1.66	2.13	1.21	2.01
May	1.96	2.72	1.85	2.41	1.82	2.13	1.37	2.22	1.22	1.97
June	2.03	3.12	2.23	2.87	2.03	2.78	2.11	2.31	1.31	2.15
July	2.09	2.89	2.08	3.23	2.02	2.31	1.96	2.35	1.27	2.23
August	2.13	3.23	2.12	2.78	2.06	2.22	2.03	2.23	1.22	2.32
September	2.08	2.51	2.06	2.36	1.83	2.05	1.38	2.07	1.31	1.99
October	1.98	2.37	1.94	2.58	1.92	2.01	1.58	2.11	1.21	2.02
November	1.92	2.48	1.89	2.34	1.82	2.52	1.28	2.15	1.25	2.01
December	1.97	2.52	1.96	2.72	1.93	2.65	2.08	2.09	1.26	2.03
Averages	2.01	2.74	1.98	2.51	1.91	2.24	1.71	2.15	1.28	2.01

. From the table observations, almost every different method is calculated in the summer (June, July, August, etc.) when the average values of $MAPE$ and Max. $MAPE$ are higher than in the other months. This result is primarily due to the more volatile weather conditions in the summer, as typhoons are common in Taiwan during the summer months. On the other hand, because electricity consumption in hot weather often constitutes record-breaking annual electricity consumption, these factors are the main reasons for the higher error value of load forecasting. As shown in

Table 3. Different methods used to measure the difference between $MAPE$ Max and .$MAPE$. Meanwhile, the hours are measured at different time points within 24 h of the “working day” (Thursday 15 October 2020) in Table 1.

Time	ANN		EP-ANN		GA-LSTM		PSO-LSTM		ILSTM-NN
	$\mathit{M}\mathit{A}\mathit{P}\mathit{E}$	$\mathbf{Max}. \mathit{M}\mathit{A}\mathit{P}\mathit{E}$	$\mathit{M}\mathit{A}\mathit{P}\mathit{E}$	$\mathbf{Max}. \mathit{M}\mathit{A}\mathit{P}\mathit{E}$	$\mathit{M}\mathit{A}\mathit{P}\mathit{E}$	$\mathbf{Max}. \mathit{M}\mathit{A}\mathit{P}\mathit{E}$	$\mathit{M}\mathit{A}\mathit{P}\mathit{E}$	$\mathbf{Max}. \mathit{M}\mathit{A}\mathit{P}\mathit{E}$	$\mathit{M}\mathit{A}\mathit{P}\mathit{E}$	$\mathbf{Max}. \mathit{M}\mathit{A}\mathit{P}\mathit{E}$
0:00	1.83	2.47	1.78	2.16	1.82	2.16	1.42	2.01	1.25	1.92
1:00	1.81	3.08	1.79	2.21	1.78	2.38	1.48	2.02	1.33	2.01
2:00	1.87	2.42	1.84	2.38	1.82	2.02	1.72	1.93	1.21	1.96
3:00	1.97	2.52	1.88	2.52	1.76	1.97	1.46	2.05	1.24	2.07
4:00	1.86	2.71	1.82	2.43	1.83	2.08	1.69	2.03	1.27	2.01
5:00	1.83	2.47	1.95	2.16	1.63	2.11	1.58	1.93	1.32	1.96
6:00	1.87	2.19	1.79	2.37	1.95	1.89	1.71	1.96	1.22	2.12
7:00	1.91	2.48	1.72	2.18	1.92	1.92	1.73	2.12	1.31	1.92
8:00	1.89	2.52	1.73	2.26	2.03	2.08	1.88	1.97	1.27	1.96
9:00	1.87	2.27	1.84	2.38	2.01	2.06	1.89	2.12	1.25	2.08
10:00	2.05	2.68	1.82	2.14	1.97	2.53	1.91	1.96	1.18	2.01
11:00	2.08	2.22	2.02	2.52	2.01	2.05	1.98	2.08	1.23	2.13
12:00	2.11	2.57	2.05	2.36	1.93	2.32	2.01	1.99	1.31	1.92
13:00	2.07	2.64	1.98	2.25	1.88	2.06	2.03	2.05	1.29	2.01
14:00	2.05	2.45	2.12	2.63	2.02	2.05	1.82	2.18	1.24	2.08
15:00	1.96	2.72	1.93	2.18	1.97	2.02	1.72	1.97	1.23	1.96
16:00	1.92	2.63	1.86	2.47	2.01	2.01	1.75	2.09	1.31	2.03
17:00	2.01	2.87	1.92	2.38	1.85	2.09	1.71	2.11	1.29	2.08
18:00	1.97	3.12	1.94	2.22	1.72	2.13	1.56	2.06	1.27	2.01
19:00	1.99	2.62	2.07	2.41	1.88	1.88	1.68	2.02	1.29	1.95
20:00	1.91	2.56	2.01	2.13	1.62	2.08	1.67	2.05	1.25	1.93
21:00	1.83	2.51	1.75	2.36	1.68	2.45	1.72	2.03	1.23	2.01
22:00	1.94	2.84	1.82	2.18	1.73	2.03	1.76	1.92	1.26	1.92
23:00	1.93	2.62	1.81	2.26	1.65	1.75	1.55	1.98	1.28	1.89
Averages	1.93	2.57	1.89	2.31	1.86	2.05	1.72	2.03	1.26	1.99

, the different $MAPE$ and Max. $MAPE$ between the hours measured at different points in time within 24 h of the “working day” (Thursday 15 October 2020) are calculated using different methods listed in Table 1. Each hour calculated at different points in time is different, including $MAPE$ and Max. $MAPE$. Observable from the table, using the ILSTM-NN method, the sum of Max. $MAPE$ and $MAPE$ per hour calculated by the method are smaller than the values calculated by the other four methods. The $MAPE$ average value calculated using the ILSTM-NN method is lower than ANN, EP-ANN, GA-LSTM, and PSO-LSTM by 53.1%, 50.0%, 47.6%, and 36.5%, respectively.

Shown in Figure 10 are the four seasons of 2020 (spring, summer, autumn, winter) calculated using the ILSTM-NN method. Comparing $MAPE$ average load forecasting for a typical week and an atypical week, as seen from the four seasons, the $MAPE$ value measured in summer is higher than the load forecasting for the other three seasons. The $MAPE$ value in summer is approximately 3%, 3.9%, and 2.3% higher than in spring, autumn, and winter. Furthermore, Max. $MAPE$ in summer is higher than spring, autumn, and winter by approximately 4.4%, 3.6%, and 5.2%, respectively. (The atypical week refers to the week when there are rainy days, typhoon days, etc.) As shown in Figure 11, the load forecasting Max. $MAPE$ for a working day (Thursday 2020/10/15) is compared with the other four methods, ANN, EP-ANN, GA-LSTM, and PSO-LSTM, and its improvement values are about 29%, 16%, 3%, and 2%, respectively.

9. Conclusions

The current short-term load forecasting model cannot handle complex nonlinear loads and the hidden deep relationships between data, resulting in the model’s subpar forecasting accuracy. In this paper, a solution to improve the LSTM-NN method for short-term load forecasting is proposed for the complexity of the power system. The CSSA method was used here to adjust the parameter values of LSTM-NN. CSSA’s superior search ability can determine the optimal parameters of LSTM-NN, and thus, we can use the LSTM-NN to generate the optimal solution of load forecasting. The advantages of CSSA include that (1) there are not many parameters to adjust, and that (2) it does not fall into the trap of finding locally optimal solutions to problems, instead finding the globally optimal solution. The improved LSTM-NN is applied to the actual load forecasting for a region. Based on our data, the $MAPE$ average is only about 2.6% to 2.8%. Compared with several other methods used usually for load forecasting (ANN, EP-ANN, GA-LSTM, and PSO-LSTM), the improved LSTM-NN significantly decreased (improved) the $MAPE$ by nearly 20% to 50%, and the average RMSE decreased (improved) by nearly 44.1% to 52.1%. The superiority and practicability of this method are clearly demonstrated in this work, rendering the improved LSTM-NN ideal for practical complex load forecasting.