Power System Dispatch Based on Improved Scenario Division with Physical and Data-Driven Features

Abstract

In power systems with high penetration of renewable energy, traditional physical model-based optimal dispatch methods suffer from modeling difficulties and poor adaptability, while data-driven dispatch methods, represented by reinforcement learning, have the advantage of fast decision making and reflecting long-term benefits. However, the performances of data-driven methods are much limited by the problem of distribution shift under insufficient power system scenario samples in the training. To address this issue, this paper proposes an improved scenario division method by integrating the power system’s key physical features and the data-driven variational autoencoder (VAE)-generated features. Next, based on the scenario division results, a multi-scenario data-driven dispatch model is established. The effectiveness of the proposed method is verified by a simulation conducted on a real power system model in a province of China.

1. Introduction

Power systems are experiencing increasing penetration of renewable energy. The strong uncertainty of renewables makes the power system dispatching problem complex. So, the research of optimal dispatching methods for power systems with high renewable energy penetration is of great importance [1,2,3,4,5].

Traditional optimization-based dispatching methods suffer from modeling difficulties and calculating complexity, which make it difficult to meet the real-time decisions of complex power systems [6,7,8]. In contrast, data-driven dispatching methods represented by reinforcement learning have the advantages of strong adaptability to environmental uncertainty, high real-time decision efficiency, and a balance between short-term and long-term benefits [9,10,11,12,13,14].

Data-driven dispatch methods can be further divided into single-scene and multi-scene dispatch. Single-scene data-driven decisions are relatively simple and widely used, but the problem of dispatching operating scenario data differences is usually ignored. Ref. [15] proposed microgrid dispatch strategies based on asynchronous dominant actor-critic (A3C) algorithms. Ref. [16] proposed a robust economic dispatch strategy for virtual power plants and ref. [17] proposed an energy storage system control strategy for a combined wind–solar storage system scenario based on deep reinforcement learning (DRL). Ref. [18] proposed an optimal dispatch method for integrated energy systems by using the proximal policy optimization (PPO) algorithm for considering security constraints.

The above data-driven dispatch methods in a single scenario usually suffer from the problem of a limited number of scenarios in the training samples, which leads to a large difference between the actual operating scenarios of the power system and the training scenarios, known as the distributional drift problem [19]. Therefore, a multi-scenario dispatch method is proposed. Ref. [20] proposed a joint optimization model for a multi-scenario microgrid with wind–photovoltaic–load storage with particle swarm optimization (PSO). Ref. [21] proposed a generation–load coordinated optimal dispatch method, considering the multi-scenario uncertainty of wind power. Ref. [22] proposed a dispatch method in multiple generation–load scenarios using the deep deterministic policy gradient (DDPG) algorithm, prioritizing operational security and efficiency. Ref. [23] aimed at reducing the wind power curtailment rate, considering different wind farms under different scenarios.

Current multi-scenario dispatch methods still have some limitations, and there are still fewer studies on how to utilize massive, high-dimensional data to better screen the features representing generation–load and give accurate decisions. Therefore, it is necessary to analyze complex scenarios and optimize scheduling under multi-scenarios.

To address this issue, this paper proposes a scenario division method for power system dispatch. Firstly, the features of massive source and load are mined according to historical data, and a data-driven feature set is constructed. Secondly, the clustering model is used to divide multiple scenarios and construct the scheduling model for each scenario. Finally, based on the intelligent decision algorithm, the unit scheduling decision is realized. The contributions are as follows:

• To address the limitations of data-driven dimensionality reduction methods, a data-driven dimensionality reduction method combined with power system physical indicators is proposed to construct system operational features. The extracted features are more interpretable and comprehensive than the traditional data-driven scenario division method.
• Based on the above integrated physical and data-driven features, the Gaussian Mixture Model (GMM)-based power system scenario division method is proposed, which can effectively identify the differences in the features of each operation section.
• Based on scenario division, a multi-scenario data-driven dispatching model using a proximal strategy optimization algorithm is proposed for the rapid decision making of power system dispatch.

The remainder of the paper is organized as follows. Section 2 proposes the framework of multi-scenario data-driven dispatch. Section 3 constructs the power system’s operational features. Section 4 proposes scenario division methods. Section 5 proposes a data-driven dispatch model of the power system. Section 6 validates the methods by simulations. Finally, Section 7 concludes the paper.

2. Framework of Power System Dispatch with Scenario Division

The framework of the proposed power system dispatch with scenario division is shown in Figure 1.

Firstly, the power system historical operation sections are obtained, which cover the operating data of thermal and renewable generations, loads, and the grid. These operation sections represent the operating states of the grid at a certain historical instant [24]. Next, the data-driven features are extracted using a variational autoencoder (VAE) based on probability distribution, and the power system’s key physical features are selected considering the generation–load features. Then, the GMM-based clustering method is used to divide the operation sections into scenarios, and the division is verified based on the Silhouette Coefficients (SC). Finally, in the offline stage, a reinforcement learning-based dispatch model is established for each divided scenario, and the scenario-specific agent is trained based on the PPO algorithm. In the online stage, the scenario-specific agent is used to realize fast online decision making for power system dispatch.

3. Power System Operational Features

The power system operational data have a massive scale, large dimension, and multiple spatiotemporal scales. A feature extraction method integrating the data-driven and physical features is proposed and shown in Figure 2, which will provide a strong basis for the operational scenario division later. As shown in Figure 2, the system level features reflect the overall operating trend of the power system, and the nodal level features reflect the overall load and generation level of the power system. Therefore, at the system level and nodal level, physical feature selection and the construction of VAE-based feature extraction models are used for key feature mining in this paper.

Among the system level, peak load and average load present the trend of load, and renewable average, renewable peak-valley difference, and renewable peak present the trend of renewables. Among the nodal level, load levels reflect the operating condition of the power system, and high load levels indicate a high demand for electricity in the system. Renewable outputs reflect the system’s renewable processing, and high renewable outputs indicate sufficient renewable energy resources.

3.1. Data-Driven Feature Extraction Using VAE

By maximizing information retention and minimizing reconstruction loss of the decoder, VAE can reduce the hidden layer feature space dimension and extract features. Thus, the negative impact of generation–load uncertainty on subsequent power system dispatch operations can be effectively eliminated. The structural diagram of the feature extraction model based on VAE is shown in Figure 3. As shown in Figure 3, the mean and variance are extracted by the decoder to fit the data distribution. Then, the dataset is reconstructed through a decoder. During the network iteration process, the network is iteratively corrected by using the reconstruction error between the original dataset and the reconstructed dataset.

As shown in Figure 3, the VAE contains an encoder and a decoder. The encoder is the inferred network $q_{\rho }(z|x)$ with parameter $\rho$, which is used to generate the variable probability distribution of the hidden variable z. The decoder is the generative network $p_{\theta }(x|z)$ with parameter $\theta$, which reduces the approximate probability distribution of the original data z. The two probability distribution density models are constructed by multilayer neural networks [25,26,27,28], which can effectively characterize the high-dimensional data of the operation section.

The specific steps of VAE-based data-driven feature extraction proposed in this paper are as follows:

(1)

Pre-processing the original power system historical operation section data so that the data satisfy a normal distribution with a mean of 0 and a variance of 1.

(2)

Using the pre-processed data as the input to the VAE model, the encoder maps the input data to the mean vector and variance vector in the latent space.

(3)

Sampling from the latent space to obtain the hidden variables containing Gaussian noise.

(4)

The decoder maps the hidden variables in Step 3 back to the original high-dimensional data space and calculates the reconstruction loss.

(5)

Minimizing the loss function using the backpropagation algorithm, updating the model parameters by stochastic gradient descent until the reconstruction loss is less than a threshold, and ensuring that the maximization evidence is under session.

The hidden variables obtained by the above steps are used as key data-driven features to characterize the high-dimensional data of the power system operation section data.

3.2. Physical Features Considering Generation–Load Features

Generation and load uncertainty is the main factor causing the difference in data distribution of the power system operation sections. The limitations of the neural network-based VAE data dimensionality reduction method are more prominent when constructing features that divide multiple scenarios. Although its powerful nonlinear transformation capability can effectively extract the data-driven features of high-dimensional data, the data-driven features only consider the complex mapping relationship from the data level, which inevitably causes a certain degree of loss of the original information. Therefore, it is very necessary to consider the key physical features that can effectively portray the generation–load features, fully explore the information related to the generation–load uncertainty in the power system, and construct more refined scenario division features to fully reflect the variability of different operation sections.

During the Chinese Spring Festival in 2023, the maximum penetration rate of renewable energy in Guangdong Province reached 30%, which is a historical peak level. Under the influence of strong cold air, renewable output increases steeply from 2600 MW to 9190 MW, causing a lack of system adjustment capacity. This example demonstrates the complexity and fast changes of operational scenarios under increasing penetration of renewable energy. So, it is necessary to select key physical features to reflect the variability of power system operation sections considering the features of generation and load. According to refs. [29,30,31,32], the key physical features that consider the generation and load features are selected as shown in

Table 1. Key physical features and implications.

Key Physical Features	Definition	Symbol
Peak load	Maximum load level	$P_{MAX}$
Average load	Average load level	$P_{AVE}$
Peak Renewable	Maximum level of renewable power output	$G_{MAX}^{NEW}$
Average Renewable	Average level of renewable power output	$G_{AVE}^{NEW}$
Renewable peak-to-valley difference	Change in the renewable power output	$G_{MAX}^{NEW}-G_{MIN}^{NEW}$

.

In this paper, the above physical features are combined with VAE-based extraction of operation section features to construct features that divide multiple scenarios. This can improve the overall performance of the data-driven dispatch model based on scenario division.

4. GMM-Based Power System Scenario Division

The data-driven features extracted by VAE and the key physical features considering the generation–load features are used as the basis for scenario division. The GMM-based scenario division method is proposed in this section. The divided scenarios will support the construction of dispatch models later.

Firstly, based on the Akaike Information Criterion (AIC), the optimal number of scenario categories considering the fitting accuracy and the number of parameters is as follows:

$$C_{\mathrm{AIC}}=2K-\ln (K)$$

(1)

Then, the GMM is used to divide the features, and the screened features are input into the GMM for scenario division; each operational scenario category and operation section with obvious differences in generation–load features can be obtained, where the probability distribution of GMM is as follows:

$$P(x)={\displaystyle \sum _{k=1}^K{\alpha }_k}\cdot p(x|{\mu }_k,{\sum }_k)$$

(2)

The number of sub-Gaussian models in the mixed model is K; $P(\cdot )$ is the probability density function; $p(x|{\mu }_k,{\sum }_k)$, ${\mu }_k$, ${\sum }_k$, and ${\alpha }_k$ are the first probability distribution model and its expectation, variance (or covariance), and model distribution weights, respectively, satisfying ${\alpha }_k\ge 0$ and ${\displaystyle \sum _{k=1}^K{\alpha }_k}=1$.

Finally, the SC [32] is used as a criterion for the effectiveness of scenario division to verify the rationality of scenario division. The contour coefficient reflects the degree of data density within the same scenario and the degree of data dispersion between different scenarios and can be used as an evaluation index to judge the effectiveness of scenario division:

$$S_{\mathrm{SC}}=\frac{b-a}{\max (a,b)}$$

(3)

where a is the average distance between the current data and other data in the same category; b is the average distance between the current data and the closest data in another category.

5. Power System Dispatch with Improved Scenario Division

5.1. Reinforcement Learning-Based System Dispatch Model

The power system dispatch problem is a sequential decision-making problem which can be modeled as a Markovian decision process and solved using reinforcement learning. The Markov decision process is constructed as follows.

5.1.1. State Space

The state space at the time instant t includes the powers of thermal units, renewable units, loads, and branches:

$$S_t=\left\{P_{G\mathrm{t}},P_{\mathrm{Lt}},Q_{G\mathrm{t}},Q_{\mathrm{Lt}},V_t^m,Los{s}_t,rh{o}_t\right\}$$

(4)

where $P_{G\mathrm{t}}$ and $Q_{G\mathrm{t}}$ are the active and reactive powers of unit G at the time t; $P_{\mathrm{Lt}}$ and $Q_{\mathrm{Lt}}$ are the active and reactive powers of the load at each node at the time t; $V_t^m$ is the voltage amplitude of each node at the time t; $Los{s}_t$ is the grid loss (network loss value) at the time t; $rh{o}_t$ indicates the actual branch current versus the rated current capacity at the time t.

5.1.2. Action Space

The agent action space $A_t$ at time instant t contains the active power $P_{Gt}^p$ for thermal units and the active power $P_{Gt}^{new}$ for renewable units:

$$A_t=\left\{P_{Gt}^p,P_{Gt}^{new}\right\}$$

(5)

5.1.3. Reward

To meet the multi-dispatch objectives of safety, reliability, economy, and environmental protection of the power system, the reward function is set as follows:

• Line-crossing limit situation (positive reward):

  $$r_{1t}=\frac{1}{n_{line}}{\displaystyle \sum _{l=1}^{n_{line}}rh{o}_{lt}}$$

  (6)

  where $n_{line}$ denotes the number of branches and $rh{o}_{lt}$ denotes the current loading rate of the branch l at the time t.
• Consumption of renewable units (positive incentives):

  $$r_{2t}=\frac{{\displaystyle \sum _{k=1}^{N_N}{P}_{Nkt}^{}}}{{\displaystyle \sum _{k=1}^{N_N}{P}_{Nkt}^{\max }}}$$

  (7)

  where $N_N$ denotes the number of renewable units; $P_{Nkt}^{}$ and $P_{Nkt}^{\max }$ denote the active output and maximum output of k renewable units at the time t, respectively.
• Balance unit power over the limit (negative reward):

  $$r_{3t}=-{\displaystyle \sum _{i=1}^{n_{balanced}}{p}_{it}}$$

  (8a)

  $$\Delta p_{it}=\{\begin{array}{l}\left|\frac{p_{it}-p_{it}^{\max }}{p_{it}^{\max }}\right|\begin{array}{cc}& \end{array}{p}_{it}^{\max }<p_{it}<1.25p_{it}^{\max }\\ \left|\frac{p_{it}-p_{it}^{\max }}{p_{it}^{\max }}\right|\begin{array}{cc}& \end{array}-1.25p_{it}^{\max }<p_{it}<p_{it}^{\max }\\ 0\begin{array}{ccc}\begin{array}{cc}\begin{array}{cc}& \end{array}& \end{array}& & \end{array}otherwise\end{array}$$

  (8b)

  where $n_{balanced}$ denotes the number of balancing units; $p_{it}$ denotes the actual active output and the maximum output of balancing units i at the time t.
• Unit operating costs (negative incentives):

  $$r_{4t}=-{\displaystyle \sum _{i=1}^{N_G}(a_i{P}_{{}_{Git}}^2+b_i{P}_{Git}+}{c}_i)$$

  (9)

  where $N_G$ denotes the total number of units, $P_{Git}$ denotes the actual active output of the unit $i$, and $a_i$, $b_i$, and $c_i$ denote the cost factor of the unit i, respectively.
• Time steps (positive reward):

  $$r_{5t}=timestep*10$$

  (10)

  where $timestep$ is the step length of the agent’s exploration.

The reward terms $r_{4t}$ and $r_{5t}$ are normalized as follows:

$$r=e^r-1$$

(11)

In summary, the domain values of the reward terms $r_{1t}$, $r_{2t}$, and $r_{5t}$ are [0, 1], and the domain values of the reward terms $r_{3t}$ and $r_{4t}$ are [–1, 0]. The total reward formula is as follows:

$$R_t={\displaystyle \sum _{i=1,2\cdots 5}{a}_i{r}_{it}}$$

(12)

where $r_{it}$ is the award of the $i$ term at time $t$, and $a_i$ is the coefficient.

The setting coefficient $a_i$ appropriately can balance the weight of each reward function and avoid overlooking certain rewards. For certain specific goals, $a_i$ can also help the agent focus on a certain type of goal and focus on learning, which can enable the agent to learn the best strategy faster. In this paper, considering the dispatch needs of safety, green, and economy, $a_i$ is set as follows:

$$a_1=1,a_2=2,a_3=4,a_4=1,a_5=1$$

(13)

where $r_{3t}$ reflects the safety of dispatch decisions, so $a_3$ is set to 4; $r_{2t}$ reflects the green environmental protection degree and economy of dispatch decisions, so $a_2$ is set to 2. In this paper, other goals are given secondary attention, so they are set to 1.

5.2. Reinforcement Learning-Based System Dispatch Model

The operational scenario-specific power system dispatch model based on the PPO algorithm is constructed in this subsection.

5.2.1. Off-Line Training

Each operational scenario agent is established and trained through continuous interaction with the power system simulation environment. The training steps are shown in Figure 4.

5.2.2. Online Decision

The execution of the dispatch takes the following steps:

• The dispatching center obtains the power system operation section state information.
• The power system operational scenario features are extracted based on the VAE, which fuses key physical features, and the GMM is used to determine the operational scenarios to which they belong.
• The scenario agent makes decisions based on the obtained power system operational state information, outputs joint actions, and issues power dispatching adjustment instructions.
• Each generating unit enters a new state after output regulation according to the schedule and at the same time, feeds the reward to the scenario agent.

6. Simulation

6.1. Simulation Setup

To verify the effectiveness of the proposed scenario division-based power system dispatching method, a real power system model in a province of China is used for validation. The system contains 748 nodes, 55 renewable units, and 187 thermal units with a dispatch interval of 15 min. 300 days of data are selected from the grid historical data, and the dataset is pre-processed and normalized before training.

The simulations are conducted in Pytorch framework on a PC with an i7-11700 CPU and 32 GB memory. The encoder and decoder of VAE use the same network structure with Relu activation function, a learning rate of 0.00001, and 9000 training rounds.

6.2. Simulation Result Analysis

(1) Feature extraction result based on VAE.

The system operation section has 303 dimensions of information (containing 55 renewable units and 248 loads). The trend of the loss function of the VAE model is shown in Figure 5. As shown in Figure 5, at the initial stage of 0–1000 iterations, the loss function gradually increases. This is because in the initial stage, the model is learning the feature representation of the data and attempting to minimize reconstruction errors. After approximately 2000 iterations, the loss function begins to converge. This means that the VAE model has learned about the potential structure and feature expression of the data. In the simulation of the paper, 48 dimensional data-driven features were extracted from 303 dimensional original data through the VAE model.

(2) Scenario division results based on GMM.

The optimal number of scenario division categories is determined to be three, based on the AIC criterion. The 300−day data are divided into three scenarios based on the GMM; the results of scenario division are shown in Figure 6. The basic idea of the GMM is to decompose data into a mixture of multiple Gaussian distribution components, where each Gaussian distribution component corresponds to a scenario. Therefore, the three types of scenarios divided by features based on the scenario division model represent the probability distribution of the three types of features. As shown in Figure 6, scenario B has relatively fewer days of scenario division compared with scenarios A and C. But the difference in score intervals among different scenarios is obvious, which effectively verifies the rationality of scenario division.

Table 2. Comparison of clustering effect based on SC values.

	Scenario Division Integrating Physical and Data-Driven Features	VAE Features
SC values	0.4522	0.0093

shows the comparison of the proposed method and the scenario division only based on VAE extraction features by using the SC values. From the table, it can be seen that the features constructed based on the method of this paper can effectively improve the division effect, and the correlation of each operation section in the operational scenario is higher.

(3) Dispatch results based on the proposed method.

To verify the effectiveness of the proposed method, the proposed method was compared with the convex optimization problem (OPT) and single scenario methods (original scenario), respectively. The comparison of results from different scenarios and methods are shown in Figure 7 and

Table 3. Comparison of dispatch time between the traditional optimization method and the proposed method.

Method	The Proposed Method	Traditional Optimization Methods
Average single-step decision time	0.003 s	19.82 s

. As shown in Figure 7, from the comparison of the mean reward of scenario A and C based on the proposed method with the original scenario, it can be seen that scenario division can effectively improve the training effectiveness of agents and accelerate agents learning efficiency. The reason why the mean reward of scenario B is lower than the mean reward of the original scenario is that there are fewer samples in scenario B, making it difficult for reinforcement learning agents to achieve optimal decision making through training. In addition, the reason why the mean reward based on the OPT is higher than the mean reward based on the proposed method is that the OPT is usually based on mathematical models and objective functions for solving and have the ability to achieve global optimization. In contrast, reinforcement learning is a method of searching through trial and error and learning, which may make it difficult to find the global optimal solution in problems with large search space and complexity. But the average single-step solving time of the OPT is 19.82 s, and the decision time of the method proposed in this paper is 0.003 s. Compared to the OPT, the decision-making efficiency of the method proposed has been improved by 99.98%. In summary, the method proposed in this paper can effectively improve the efficiency of the reinforcement learning agent’s training and decision making.

To further validate the advantages of the proposed method, a comparison was made between the proposed method and the method based on VAE. The comparison results are shown in Figure 8. As shown in Figure 8, the mean reward values of the method proposed in this paper are significantly higher than the method based on VAE during the agent training process in both scenario A and scenario C. Compared to the method based on VAE, the mean reward value of the proposed method in scenario A is increased by 43.51% and that in scenario C is increased by 35.3%. Overall, the method proposed in this paper can prevent agents from making decisions directly on the dispatch scenarios with uncertainty distribution and reduce the learning difficulty.

Based on the analysis of power system dispatch results, the output power dispatch results of renewable energy with and without scenario division are shown Figure 9. The renewable energy consumption rate refers to the ratio of the actual output power of renewable energy used by the system to the total output power of renewable energy at the current time. As shown in Figure 9, the renewable energy consumption rate of nodes 13–22 with scenario division are significantly improved compared to without scenario division. Overall, the decision results of the proposed method are better in renewable energy consumption.

From the perspective of operating costs in the power system dispatch, unit operating cost with and without the scenario division are shown in Figure 10. As shown in Figure 10, the unit operating cost without the scenario division is USD 1,335,901, and the unit operating cost after scenario division is USD 1,266,477. The overall power system operating cost is reduced by 5.196%, and the peak of the unit operating cost is reduced.

7. Conclusions

A multi-scenario data-driven dispatch method based on scenario division integrating VAE-generated and physical features is proposed and verified by simulations.

To address the power system dispatch problem under multiple and fast-changing operational scenarios, firstly, VAE is used to extract the potential data-driven features, which can reduce the redundancy of high-dimensional features and accurately describe the operation features of generation–load data to the greatest extent. At the same time, the key physical features are selected, considering the features of generation–load, which are interpretable and can screen the operation features from the overall level and changing trend of generation–load. This provides a more comprehensive and interpretable feature. Based on these features, the GMM is used to divide the scenarios and distinguish the scenarios with obvious differences in generation–load operation features. Using SC to evaluate the effectiveness of scenario division. Finally, the dispatching model of each scenario is constructed and the precise dispatching decision under each scenario is given. Compared with a single scenario dispatching decision, this method is more accurate and reasonable for the actual dispatching requirements.