Operation Optimization Model of Regional Power Grid Considering Congestion Management and Security Check in Complex Market Operation Environment

Abstract

Security checks are essential for ensuring the safe operation of the regional power grid (RPG) and the smooth functioning of the electricity spot market (ESM). Currently, China’s RPG operating environment encompasses a complex mix of centralized ESM, decentralized ESM, and planned power generation. This complexity has led to increasingly severe RPG congestion issues. To address this, this paper introduces a security check mechanism design and operational optimization approach tailored for RPGs in complex markets, with a focus on congestion management. Firstly, the paper elaborates on the practical foundations, unique constraints, and requirements for security checks and congestion management during the RPG’s operational mode transitions. Secondly, it outlines the principles underlying the security check mechanism and presents a framework for RPG security checks and congestion management. Through a comparative analysis of three different programs, including their advantages, disadvantages, and applicable scenarios, the paper provides an optimal program recommendation. Building on this, the paper develops an operational optimization method that incorporates congestion management for each of the three security check and congestion management programs. Lastly, an IEEE-39 node test system is simulated to validate the effectiveness of the proposed programs. The mechanism and simulation analysis results show that Program 3, based on market mechanisms, has theoretical and practical advantages over Program 1 (based on multiple adjustments) and Program 2 (based on dispatch plans) for congestion management. Under the same line congestion situation, Program 1 requires two adjustments to relieve the line congestion, while Program 2 and Program 3 can solve the problem with just one optimization adjustment, and the congestion management effect of Program 3 is more obvious and superior.

1. Introduction

1.1. Background and Motivation

With the growth of global energy demand and the intensification of environmental protection pressure, energy transformation has become an inevitable trend [1,2]. With this background, the power system reform in China came into being, aiming to promote the optimization of energy structure and the green development of the economy by building a more efficient, clean, and sustainable power market system [3,4,5]. In recent years, the electric power system reform in China has made remarkable progress, especially the reform measures that started with the construction of the provincial ESM. Since 2017, China has launched the first batch of ESM pilots, and eight provinces have been selected to try. By 2020, the second batch of six ESM pilots will be launched one after another and will enter the trial operation stage of continuous settlement. With the deepening and advancement in the pilot work, some provinces have successfully entered the formal operation stage of the ESM [6,7,8]. In 2022, the China government further put forward a strategic plan to build a unified national electricity market, which aims to form a multi-level power market operation mode with a provincial market, regional market, and national market by integrating power market resources at all levels, to realize the optimal allocation and efficient utilization of power resources [9].

The global development of ESM presents a diversified pattern, and different countries have formed their own market systems according to their political, economic, and geographical characteristics. The United States, Northern Europe, and the United Kingdom (UK) are regarded as typical representatives of the spot electricity market, as shown by their unique operation modes and market structures. Currently, there is no directly applicable inter-provincial spot trading scheme available internationally. Specifically, the market scale and grid structure of the Pennsylvania-New Jersey-Maryland (PJM) electricity market in the United States [10,11] are similar to that of the southern regional electricity market. However, the states in the United States have a relatively high level of independence, with most electricity supply and demand being balanced locally, and there is not a high demand for wide-ranging energy allocation. Therefore, the PJM market is operated and dispatched by a single system operator without a multi-level dispatching structure, and a cross-regional transmission and trading system has not been established [12,13]. The regional electricity market in Northern Europe [14,15] has established a unified optimization clearance mechanism for cross-border electricity spot trading. However, since the congestion in the Nordic power system mainly occurs between cross-border and cross-regional areas, their clearance model does not consider fine-grained flow constraints within each region [16], making it unsuitable for the regional electricity market with severe congestion in China. Since 2005, the UK electricity spot market has undergone major reforms, aiming at creating a unified and efficient electricity trading system [17]. These reforms include implementing a unified transaction, balance, and settlement mechanism, unifying transmission pricing and power grid use rights contracts, and promoting market competition and the development of renewable energy. These measures have significantly reduced operating costs, broken market monopoly, and enhanced market competitiveness. At the same time, the reform has promoted the efficient utilization and cost reduction of renewable energy and promoted the development of the UK electricity market in a more open, efficient, and sustainable direction [18,19]. In summary, the PJM market in the United States, the Nordic power market, and the UK power market each have their own unique characteristics in terms of operational mechanisms, market structures, and trading modes. The PJM market emphasizes centralized trading and nodal pricing, the Nordic market focuses on cross-border trading and market integration, while the UK market relies primarily on bilateral contracts supplemented by centralized trading. All three reflect the reform and development paths of power markets in different countries and regions. Therefore, the structural design of a unified spot electricity market system in the United States, Northern Europe, and the United Kingdom with leading power markets is closely related to their own institutional boundaries and cannot be copied. Because there are significant differences between Europe and the United States in the power industry system, market construction motivation, and objectives, especially in the power dispatching management system and other border conditions, it is unacceptable to simply learn from its structural design scheme in China. In a word, the above-mentioned typical mature electricity market construction experience is not consistent with the actual situation of regional electricity market construction in China.

With the rapid development of ESM construction in China, the formation mode of power generation plans in various provinces in the regional power grid (RPG) has undergone a major change, showing a complex form. Specifically, the RPG is generally composed of multiple provincial power grids, including multiple voltage levels, and belongs to the same frequency control area. In the past, the RPG and the provincial power grid used a set of planned, iterative, and collaborative methods to prepare the power generation plan, thus ensuring the safe operation of the RPG. However, the previous operation mode has undergone a major change. On the one hand, the market model is in parallel with the planning model; that is, some provinces with ESM have formed power generation plans by market mechanism, while some provinces without ESM still prepare power generation plans in the way of past plans. On the other hand, centralized and decentralized market models coexist; that is, in the provinces with ESM, different ESM provinces may adopt different centralized or decentralized market models.

In the above complex operation and market environment, the security check and congestion management of RPG have become the key problems that need to be solved urgently. The inter-provincial power flows in RPG affect and cross each other, and the clearing results of provincial ESM can only consider the power balance within their own scope but cannot take into account the cross-provincial power flows of neighboring provinces and cannot consider the startup mode, power generation output, and load distribution of neighboring provinces as a whole, which has serious security risks. When the provincial ESM clearance results are executed, the provincial ESM clearance results may not be executed due to the influence of the crossing tidal current changes in neighboring provinces, which may lead to the transmission tidal current exceeding the limit in the provinces. The above-mentioned factors lead to the severe challenge of RPG safe operation. Therefore, solving safety check and congestion management of RPG in a complex environment and ensuring the safe operation of the power system have become key urgent problems that need to be solved in order to promote the construction of ESM.

1.2. Literature Review

With the large-scale integration of renewable energy into the power grid and the gradual development of the electricity market, the uncertainty in power grid operation has significantly increased. Traditional economic and safety management methods are no longer sufficient to meet current demands. Therefore, it is imperative to explore grid security checks and congestion management approaches that are suitable for the new complex environment, ensuring the safe and efficient operation of the power grid.

Security check refers to the process of power grid security analysis and checks for power generation plan and real-time dispatching instructions arranged by power market clearing or planning and dispatching in order to ensure the safe and stable operation of the power system. At present, significant progress has been made in the research of power grid security checking methods. On the one hand, the traditional static security checking methods, such as ground state checking and N-1 security checking of 96 sections [20,21], predict the accident of the power grid by means of planned power flow calculation, static security analysis, short-circuit current calculation and sensitivity analysis, check its ability to withstand the accident and give preventive measures or correction measures that should be taken after the accident. On the other hand, with the development of intelligent dispatching systems, the method of power grid security checks is constantly innovating. For example, based on the regional whole network model [22], the provincial power spot market optimization clearing model and the whole network security check method, through the whole network sensitivity calculation and optimization clearing and the whole network security check iteration, accurate power flow and node electricity price are obtained, which improves the accuracy and effectiveness of power grid security check. Cai Z et al. [23] propose a novel closed-loop security-constrained unit commitment framework to acquire the global optimal solution by integrating the N-1 security check in the unit commitment directly, which effectively improves the security and economic performance of generation scheduling. Considering power quantity constraints and the future state grid model, Wu D et al. [24] construct a power quantity margin solution model, and the medium and long-term electric quantity security check and adjustment are realized through the correlation coefficient matrix. Aiming at the issue of security analysis on medium and long-term electricity transactions in China, Li Z et al. [25] propose a step-by-step security analysis method combining monthly elastic pre-checking and day-ahead security analysis. According to the characteristics of medium- and long-term transactions, Liu S et al. [26] put forward a linear optimization model to check the security of medium- and long-term transactions and carry out security checks and congestion management while determining the winning bid. Zhang Y et al. [27] propose an online frequency security assessment based on an analytical model considering limiting modules, which implements security assessment by calculating and checking the frequency features. Lu Y et al. [28] analyze the content of security checks and the factors that affect their accuracy and propose a refined security check scheme in the provincial power grid for electricity market operation. The aforementioned studies have all established security check mechanisms and operation optimization methods for a single electricity market, designing relevant optimization models and solution algorithms. However, there is currently no RPG security check method or model that addresses multiple types of market operations, and thus, they cannot solve the problem of ensuring RPG’s safe operation after the superposition of clearing results from multiple provincial electricity markets.

Scholars and engineers have made rich achievements in the research of congestion management in the electricity market environment. On the one hand, some studies focus on using market means such as price signals or contracts to influence flexible demand behavior, thus alleviating the transmission congestion problem. These studies not only provide new ideas for congestion management but also provide theoretical support for the operation of the electricity market. On the other hand, some studies pay more attention to how to formulate effective congestion management strategies in combination with the actual situation of the electricity market to ensure fairness, justice, and openness in the power market. In Literature [29], the main design challenges of local flexibility markets for congestion management are identified and discussed based on desirable market properties from economics theory. Holst B et al. [30] provide the first formalization of capacity restriction contracts and redispatch contracts for congestion management and present an optimization model to guide system operators in allocating a budget between these contracts. Christy J et al. [31] provide a novel data-driven adaptive Lyapunov function with a graphical deep convolutional neural network regulated optimal flow by accurate energy management to resolve congestion management issues for smart grid. Sergio P et al. [32] introduce a local flexibility market with a multi-layered taxonomy focused on congestion management services. Rahman A et al. [33] present a chance-constrained optimization framework for transmission congestion management in the presence of wind turbines and electrical energy storage (EES) systems, which can evaluate the performance of the congestion management model in both normal and emergency conditions. Mehdi A et al. [34] present the findings on market-based congestion management in power systems. The main idea is to unlock flexibility from both small and large-scale resources by creating a platform that allows flexibility to enter the markets and be used by system operators for congestion management. Abhimanyu K et al. [35] present a probabilistic security-constrained optimal power flow (SCOPF) model for congestion management, considering power flow controlling devices based on the non-linear AC formulation. Ehsan D et al. [36] present a new congestion management model developed based on power system partitioning. Using the proposed method, power system operators can put their focus on the candidate zone and alleviate the congestion effectively by some remedial measures such as generation rescheduling and load shedding. Ehsan D et al. [37] introduce a new congestion index to form the flexible congestion control zone and present a new dynamic zonal congestion management model based on the demand response program using power tracing techniques and sensitivity analysis. The aforementioned studies address grid congestion issues through market price signals, participation of flexible resources, and collaborative optimization. However, there is hardly any research examining the RPG congestion management problem arising from the superposition of different market clearing results, and the related congestion management models do not consider the constraint relationship between RPG security checks and congestion management optimization.

Generally speaking, the existing research on power grid security checks mainly includes four aspects. (1) Aiming at the single market form, the security check framework of dispatching plan is put forward; (2) Power grid security checking method for medium and long-term electricity trading is established; (3) The model and algorithm of power grid security check for planned dispatching and economic dispatching are put forward; (4) In the provincial ESM construction, the coordination mechanism with RPG security check is not considered in view of the power grid security check mechanism design program under the single market form in this province. However, there is no security checking method or congestion management mechanism that matches the current construction process of the electric power spot market and the multi-level dispatching coordination mechanism of China.

1.3. Contribution and Innovation

To this end, according to the actual demand for the economic and safe operation of RPG and the deficiency of existing research, this paper designs the security check mechanism and operation optimization method of regional power grid considering congestion management in complex ESM. The main contributions of this paper are as follows:

(1)

The realistic foundation, special constraints, and needs of security check and congestion management existing in the operating mode changing process of the RPG are expounded. Considering the centralized power market, decentralized power market, fixed power generation plan, and complex operation environment in RPG, the status and challenges of the transition from planned scheduling mode to market-based scheduling mode are analyzed. The basic rules of regional power grid operation and the root cause of power grid congestion in the operation of regional power grids are also explained through a detailed modeling process.

(2)

Security checks and congestion management programs of the regional power grid are designed. Facing the specific reasons for congestion in the regional power grid, the principles of the security check mechanism and the framework of RPG security check and congestion management mechanism are put forward. In the three designed programs of security check and congestion management, the operation optimization logic of the regional power grid and the transmission relation of dispatching information is clearly designed, which considers three typical market models, namely, provinces without ESM, provinces with centralized ESM, provinces with decentralized ESM. Based on these, the best program suggestion is given by comparing and analyzing the advantages, disadvantages, and applicable scenarios of each program.

(3)

Operation optimization methods that consider congestion management are established for three different security checks and congestion management programs. The operation optimization method established aims to minimize power adjustment and re-adjustment costs of RPG while taking into account existing provincial ESM clearing results. Based on the congestion management model, an IEEE-39 node test system is simulated as an example to verify the effectiveness of the three proposed programs for day-ahead safety check and congestion management.

2. System Structure and Problem Statement

2.1. System Structure

The RPG studied in this paper is composed of provincial power grids with different ESM operation modes and scheduling schemes, namely, provinces with centralized ESM, provinces with decentralized ESM, and provinces without ESM. Figure 1 shows the system structure of the regional power grid.

The dispatch center of RPG is the executor and decision maker of RPG power grid operation. According to the market clearing results of provinces with centralized ESM and provinces with decentralized ESM, as well as the planned scheduling result of provinces without ESM, the dispatch center of RPG conducts safety check and congestion management on the operation plans of provincial power grids and supervise and guide the safe operation of provincial power grids.

For the provinces with ESM, the market participants include wind power, photovoltaic, hydropower (HP), and coal-fired power (CFP), as well as power wholesale users and power-selling companies. As the operator of the provincial power grid, the dispatching center of the provincial power grid carries out the provincial power grid operation plan and the provincial market transaction decision under the guidance of the dispatching center of RPG.

For the provinces without ESM, which includes power generation units and load types similar to provinces with ESM in their power grids, the difference is that they execute a fixed power generation plan and do not participate in the operation of the electricity market.

2.2. Problem Statement

As mentioned earlier, each province within the regional power grid is located within the same frequency control zone. The dispatch center of RPG conducts safety checks based on the market clearing results of provinces with ESM and the planned dispatch results of provinces without ESM according to the constraints of regional power grid line flow. Based on the above relationship, this paper focuses on how to effectively solve the security check and congestion management problems of RPG under the constraint of line security capacity and puts forward feasible operation optimization mechanisms and models for different market operation modes in RPG.

Therefore, this paper analyzes the causes of congestion in RPG under the market environment firstly, which is the basis of security check mechanism design and congestion management modeling. For the complex ring power grid structure shown in Figure 1, the power flow constraint of the overall view system in the same frequency control area cannot be deduced from the set of power flow constraints of each subspace view system. Besides, the mathematical modeling of power flows in complex electrical networks is the basis of safety check mechanism design and congestion management modeling for RPG, and the related research has been relatively mature, and this paper analyzes it with reference to the mature power flow model, which can be referred to references [38,39,40]. This section takes IEEE-39 node topology [38] as an example to analyze the causes of congestion management in RPG. This section divides the IEEE-39 node system into three subspace structures: structure A, structure B, and structure C, as shown in Figure 2.

From the perspective of each subspace, the power flow constraint of the system line [39,40] in the ESM clearing model is expressed as follows.

$${\omega }_A:-F_{LA}^{\max }\le G_{LA-IA}{P}_{IAt}+G_{LA-NA}{D}_{NAt}+G_{LA-JA}P{T}_{JAt}\le F_{LA}^{\max }$$

(1)

$${\omega }_B:-F_{LB}^{\max }\le G_{LB-IB}{P}_{IBt}+G_{LB-NB}{D}_{NBt}+G_{LB-JB}P{T}_{JBt}\le F_{LB}^{\max }$$

(2)

$${\omega }_C:-F_{LC}^{\max }\le G_{LC-IC}{P}_{ICt}+G_{LC-NC}{D}_{NCt}+G_{LC-JC}P{T}_{JCt}\le F_{LC}^{\max }$$

(3)

where $I$ is the generator set; $N$ is the load set; $J$ is the cross-regional tie line set; $L$ is the transmission line set; $P$ is the generator power; $D$ is the load power; $PT$ is the tie line exchange power; $G$ is the power transfer distribution factor; $F_L^{\max }$ is the upper limit of line transmission capacity.

From the overall spatial perspective of the same frequency control area, the line power flow constraints of the system are expressed as follows.

$$\omega :-F_L^{\max }\le G_{L-I}{P}_{It}+G_{L-N}{D}_{Nt}\le F_L^{\max }$$

(4)

If and only if the switching power settings of tie lines in three subspaces A, B, and C before spot market optimization are consistent with the switching power in the final actual operation, ${\omega }_A\cap {\omega }_B\cap {\omega }_C\Rightarrow \omega$ will be established. However, the above conditions need to take the post-operation results as the pre-optimization input, and the logic is inverted, which is not valid in time sequence and technology.

Based on the above analysis, it can be concluded that the design of separated subspaces will inevitably lead to congestion problems due to the deviation in system power flow constraints in the complex ring power grid belonging to the same frequency control area. If the transmission capacity of the line is reduced through each subspace to avoid ring network congestion, the transmission efficiency of the line will be sacrificed, which is contrary to the economic operation of the system and the original intention of interconnection line construction.

In summary, the issue of security checks has become a constraint on the construction of a unified electricity spot market in China, and it is necessary to design the security check and congestion management program for the regional power grid, considering the complex market situation and power grid structure within RPG.

3. Security Check Mechanism Design of Regional Power Grid

3.1. Principles of Security Check Mechanism Design

In the complex market situation and power grid structure, the security check for RPG involves a series of work collaborations and cooperations between planned scheduling and market mechanisms, as well as between the regional dispatch center and provincial dispatch center on a time scale. The security check of RPG must be coordinated with the clearing link of the provincial market recently to ensure the timely clearing of the provincial market and treat the security check of power generation plans of provinces with ESM and provinces without ESM fairly. In view of this, the framework design of the RPG security check mechanism in a complex market environment should follow the following basic principles.

(1) Safe and stable operation of RPG must be guaranteed. Ensuring the safe operation of RPG in the complex electricity market environment is the starting point and the purpose of security checks and congestion management mechanism design for RPG, and it is also a prerequisite for promoting the development of the electricity market.

(2) The generation plans and market clearing results of the provincial power grid within the RPG must be considered in the security check process of the RPG. Due to the establishment and operation of the provincial ESM, if the original operation generation and market clearing results of each provincial power grid are not included, it will render the results of the provincial clearing meaningless and lead to significant safety operational issues. Generally, in the process of designing a security check mechanism for RPG, minimal adjustments should be made to the generation plans formed through planning or market means by each province, and their unit combination result should generally remain unchanged. However, if it affects the safe and stable operation of RPG, the dispatching center of RPG has the right to adjust the unit combination of each province.

(3) The clearing process and time sequence of ESM in pilot provinces need to be coordinated efficiently. On the basis of giving consideration to the market organization process of provinces with different ESM modes and the planning process of provinces without ESM, a safe, efficient, and feasible safety check connection and cooperation scheme between RPG and the provincial power grid is put forward.

3.2. Framework Design of RPG Security Check and Congestion Management Mechanism

The primary tasks of security check and congestion management for RPG include safety inspection of the day-ahead dispatching plan of each province and identification of the potential power flow excess on the transmission section in RPG to facilitate the subsequent transmission congestion management. Due to the existence of two-level dispatch institutions in RPG and the provincial power grid, different programs can be adopted for security checks and congestion management. Considering the different principles of adjusting the dispatching plan in different provinces, the security check and congestion management mechanism design program can be divided into three types.

3.2.1. Framework Design of Program 1

In Program 1, the dispatching center of RPG will conduct safety checks of the regional power grid on the basis of the provincial day-ahead dispatching plans, which are prepared by planning or formed by the market mechanism. Figure 3 shows the framework of security check and congestion management of Program 1, and

Table 1. Information flow in Program 1.

Input Information for Safety Check and Congestion Management
Dispatching center of RPG	⟡ Stability power limit of RPG
	⟡ Power grid interface operation plan of tie line
Dispatching center of the provincial power grid	⟡ Power grid topology information
	⟡ Unit parameters
	⟡ Day-ahead pre-dispatch plan (Provinces without ESM) ⟡ Initial pre-dispatch plan (Provinces with centralized ESM) ⟡ Pre-dispatch plan for accumulating market transaction results in day-ahead ESM (Provinces with decentralized ESM)
	⟡ Short-term system load forecasting ⟡ Short-term bus load forecasting
Output information for safety check
Dispatching center of RPG	⟡ Cross-line information of RPG
	⟡ Sensitivity analysis of congestion lines
	⟡ Adjustment information of ⟡ Adjustment information of generator operation plan in the provincial power grid *
	⟡ Adjustment information of tie-line gateway plan *

Note: The information marked with * in the table is the output information when the provinces cannot solve their own congestion.

shows the information flow in Program 1.

When the safety check fails, the dispatching center of RPG will release the line crossing information to the relevant provinces. On this basis, the dispatching center of provinces adjust their day-ahead dispatching plans and report them again for re-checking. If the dispatching center of provinces cannot solve the problem of RPG congestion by the above methods, the dispatching center of RPG has the priority of adjusting the tie-line gateway and the unit power generation plan and managing the RPG congestion based on the unit sensitivity.

3.2.2. Framework Design of Program 2

In Program 2, the dispatching center of RPG will conduct safety check and congestion management in a planned way on the basis of the provincial day-ahead dispatching plans, which are prepared by planning or formed by market mechanism. Figure 4 shows the framework of security check and congestion management of Program 2, and

Table 2. Information flow in Program 2.

Input Information for Safety Check and Congestion Management
Dispatching center of RPG	⟡ Stability power limit of RPG
	⟡ Power grid interface operation plan of tie line
Dispatching center of the provincial power grid	⟡ Power grid topology information
	⟡ Unit parameters
	⟡ Day-ahead pre-dispatch plan (Provinces without ESM) ⟡ Initial pre-dispatch plan (Provinces with centralized ESM) ⟡ Pre-dispatch plan for accumulating market transaction results in day-ahead ESM (Provinces with decentralized ESM)
	⟡ Short-term system load forecasting ⟡ Short-term bus load forecasting
Output information for safety check
Dispatching center of RPG	⟡ Cross-line information of RPG
	⟡ Sensitivity analysis of congestion lines
	⟡ Adjustment information of ⟡ Adjustment information of generator operation plan in the provincial power grid
	⟡ Adjustment information of tie-line gateway plan

shows the information flow in Program 2.

When the safety check fails, the dispatching center of RPG takes the minimum adjustment of the system as the objective function, adjusts the day-ahead power generation plans of the provinces according to the sensitivity of each unit to blocked lines, and releases the adjustment results to the provinces, which will cooperate with each other according to the released information.

3.2.3. Framework Design of Program 3

In Program 3, the dispatching center of RPG will conduct safety check and congestion management of RPG by the market mechanism on the basis of the provincial day-ahead dispatching plans, which are prepared by planning or formed by the market mechanism. Figure 5 shows the framework of security check and congestion management of Program 3, and

Table 3. Information flow in Program 3.

Input Information for Safety Check and Congestion Management
Dispatching center of RPG	⟡ Stability power limit of RPG
	⟡ Power grid interface operation plan of tie line
Dispatching center of the provincial power grid	⟡ Power grid topology information
	⟡ Unit parameters and operation plan
	⟡ Short-term system load forecasting ⟡ Short-term bus load forecasting
Output information for safety check
Dispatching center of RPG	⟡ Cross-line information of RPG
	⟡ Sensitivity analysis of congestion lines
	⟡ Adjustment information of ⟡ Adjustment information of generator operation plan in the provincial power grid

shows the information flow in Program 3.

Based on the quotations of each market agent, the dispatching center of RPG takes the minimum total adjustment cost of the system as the objective function to optimize and readjust the provincial dispatching plans, considering the constraints of power grid security and unit characteristics. In Program 3, the temporary demand for peak shaving and peak electricity energy in various provinces can also be solved through the above market mechanism.

3.3. Program Characteristic Analysis

The comparison of the characteristics of the three proposed programs for safety check and congestion management for RPG is shown in

Table 4. Difference between the three security check and congestion management programs.

Programs	Compliance Degree with Provincial Dispatching Plans	Mechanism Operation Efficiency	Fairness and Resource Allocation Efficiency
Program 1	Strong	Low	Weak
Program 2	Weak	High	Weak
Program 3	Strong	High	Strong

.

In comparison, Program 1 fully adheres to the generation plans and market clearing results of each province, but it is inefficient in some scenarios and has a weak promotional effect on optimizing the allocation of power resources among provinces within the RPG. In the case of congestion, Program 1 requires at least two rounds of day-ahead plan submissions from each province and security check by the dispatching center of RPG. Since each province cannot take into account the dispatch plans of other provinces during its own adjustment process, it may ultimately result in situations where provinces cannot resolve RPG congestion on their own, leading to repeated iterations between the RPG dispatch institution and provincial dispatch. As Program 1 primarily manages congestion from the perspective of each province, it has a weak promotional effect on optimizing the allocation of power resources within the RPG.

In Program 2, the dispatching center of RPG and the dispatching center of the provincial power grid cooperate efficiently. Based on adhering to the generation plans and market clearing results of each province, it makes fine adjustments to achieve congestion management, but it lacks sufficient promotion of optimizing regional resource allocation. The core of Program 2 is a planned scheduling method that combines safety check and congestion management, avoiding multiple iterations and cooperation among various levels of dispatching centers. It has strong operability in the early stages of complex electricity market environments. However, due to the direct implementation of congestion management by RPG, it does not fully reflect the initiative and willingness of each province to eliminate transmission congestion of RPG and does not play the role of market mechanisms.

In Program 3, the market mechanism is adopted to address the collaborative issues of the safety check and congestion management of RPG, taking into account multiple dimensions such as practical foundations, efficiency, and fairness of cooperation among various levels of dispatching centers and promoting optimal allocation of resources. The security check mechanism proposed in Program 3 can be efficiently integrated with the current inter-provincial valley peak shaving market mechanism in various regions and improved into a day-ahead regional regulation market trading mechanism that includes both the increase and decrease in system power generation. Market clearing can solve the valley peak shaving problem, peak energy shortage problem, and congestion management problem of RPG faced by each province, which is conducive to the optimization of power resource allocation in the RPG in advance.

Combined with the comparison indexes in Table 4 and the above analysis results, Program 3 has advantages in compliance with provincial dispatching plans, mechanism efficiency, expense allocation, promoting optimal allocation of the RPG, and is more advantageous in the economic and safe operation of RPG compared with Program 1 and Program 2. From a practical application perspective, Program 3 should be used as the ideal mode for the safety check and congestion management of RPG in complex electricity market environments. However, considering that the market construction and other related work of this scheme still require time, Program 2 can be adopted as a phased transition mode.

4. Operation Optimization Model Considering Congestion Management

Based on the above results of the security check mechanism design of RPG, this section establishes operation optimization models considering congestion management for three programs.

4.1. Optimization Method for Program 1 and Program 2

The optimization strategy of Program 1 is that the dispatching center of RPG will release specific information on failed safety checks to the corresponding provinces, and each province will manage regional grid congestion by adjusting its own pre-dispatch plans. In cases where provinces are unable to resolve congestion issues, the RPG has the authority to adjust the tie-line gateway and unit plans. The optimization strategy of Program 2 is that the dispatching center of RPG is responsible for security check and congestion management, with cooperation from various provinces and municipalities in implementation. The mathematical model for congestion management is identical to that in Program 1, with the distinction lying in the different scenarios of application.

4.1.1. Objective Function

The objective function of Program 1 and Program 2 is to minimize the power adjustment of the final output power of the unit.

$$\min \hspace{1em}\hspace{1em}{\displaystyle \sum _{i=1}^N\left|P_i-P_i^0\right|}$$

(5)

where N is the total number of startup units, $P_i$ is the final output power of unit i, and $P_i^0$ is the pre-dispatch plan of unit i.

4.1.2. Constraint Conditions

(1)

Power balance constraints

In the operation of regional power grids, the dispatch center must first ensure power supply to users within the RPG to ensure system power balance [41].

$${\displaystyle \sum _{i=1}^N{P}_i}+{\displaystyle \sum _{m=1}^{NT}{T}_m}=D$$

(6)

where $T_m$ is the planned power of cross-regional tie-lines m, $NT$ is the total number of cross-regional tie-lines in RPG, and $D$ is the power load of RPG.

(2)

Rotation reserve constraints

In the operation of regional power grids, the sum capacity of upward adjustment and downward adjustment of the unit should meet actual operation demand for upward and downward adjustment of the rotary standby [42,43,44].

$${\displaystyle \sum _{i=1}^N\min }\left\{\Delta P_i^U,P_i^{\max }-P_i\right\}\ge \Delta S{R}^U$$

(7)

$${\displaystyle \sum _{i=1}^N\min }\left\{\Delta P_i^D,P_i-P_i^{\min }\right\}\ge \Delta S{R}^D$$

(8)

$${\displaystyle \sum _{i\in {\Omega }_{\mathrm{z}}}\min }\left\{\Delta P_i^U,P_i^{\max }-P_i\right\}\ge \Delta S{R}_z^U$$

(9)

$${\displaystyle \sum _{i\in {\Omega }_{\mathrm{z}}}\min }\left\{\Delta P_i^D,P_i-P_i^{\min }\right\}\ge \Delta S{R}_z^D$$

(10)

where $\Delta P_i^U$ and $\Delta P_i^D$ are the maximum upward climbing rate and the maximum downward climbing rate of the unit, respectively. $P_i^{\min }$ is the minimum output of the unit, $P_i^{\max }$ is the maximum output of the unit, and ${\Omega }_z$ is the set of generator sets. $\Delta S{R}^D$ and $\Delta S{R}_z^D$ are the backup requirements for RPG and the provincial power grid, respectively.

(3)

Unit output power constraints

The real-time operation power of the unit should be within its maximum/minimum output power range.

$$P_i^{\min }\le P_i\le P_i^{\max }$$

(11)

where $P_i$ is the final output power of unit i, $P_i^{\min }$ is the minimum output of the unit, and $P_i^{\max }$ is the maximum output of the unit.

(4)

Unit climbing constraints

When the unit climbs uphill or downhill, it should meet the climbing rate requirements.

$$P_i-P_i^{t-1}\le U{R}_i$$

(12)

$$P_i^{t-1}-P_i\le D{R}_i$$

(13)

where $P_i$ is the output power of unit i in the previous period. $U{R}_i$ and $D{R}_i$ are the upward and downward climbing abilities of unit i. $U{R}_j$ and $D{R}_j$ are the upward and downward climbing abilities of unit j, respectively.

(5)

RPG lines power flow constraints

The line transmission capacity of RPG is limited, and when the power flow exceeds the rated capacity of the transmission line, it may lead to overload operation of the RPG [45,46].

$$-F_l^{\max }\le {\displaystyle \sum _{i=1}^N{G}_{l-i}{P}_i}+{\displaystyle \sum _{m=1}^{NT}{G}_{l-m}{T}_m}-{\displaystyle \sum _{k=1}^{NK}{G}_{l-k}{D}_k}\le F_l^{\max }$$

(14)

where $F_l^{\max }$ is the power flow transmission limit of the RPG line l, $G_{l-i}$ is the distribution factor of generator output power transfer from the node where the unit is located to the line l, and $G_{l-m}$ is the distribution factor of generator output power transfer from the node where tie-line m is located to the line l. $NK$ is the number of system load nodes, $G_{l-k}$ is the distribution factor of generator output power transfer from node $k$ to line l, and $D_k$ is the bus load value of the node $k$.

4.2. Optimization Method for Program 3

The optimization strategy of Program 3 is to readjust the pre-dispatch plans of the provincial power grid by the dispatching center of RPG. In this strategy, there are two options for the congestion management model. Program 3-1 is the optimization and re-adjustment aimed at minimizing adjustment costs, while Program 3-2 is the market coupling and re-clearing aimed at minimizing power generation costs.

4.2.1. Optimization Model of Program 3-1

(1)

Objective function

In Program 3-1, the dispatching center of RPG conducts safety checks on RPG based on the pre-dispatch plans of the provincial power grid and implements congestion management through market-oriented re-adjustment. The model optimizes and readjusts the pre-dispatch plans with the objective function of minimizing adjustment costs. It does not change the unit combination results of pre-dispatch plans of the provincial power grid but only changes the unit output.

$$\min \hspace{1em}{\displaystyle \sum _{i=1}^{NI}\left[{\displaystyle \sum _{b=1}^{NB}\left(\Delta P_{i,b}^{+}{C}_{i,b}^{+}+\Delta P_{i,b}^{-}{C}_{i,b}^{-}\right)}\right]}+{\displaystyle \sum _{j=1}^{NJ}\left[\Delta P_j^{+}{\lambda }_j\left(P_j^{rt}\right)+\Delta P_j^{-}{\lambda }_j\left(P_j^{rt}\right)\right]}$$

(15)

$$P_i^{rt}=P_i^0+{\displaystyle \sum _{b=1}^{NB}\left(\Delta P_{i,b}^{+}+\Delta P_{i,b}^{-}\right)}$$

(16)

$$P_j^{rt}=P_j^0+\Delta P_j^{+}+\Delta P_j^{-}$$

(17)

where $NB$ is the total number of segments of the upward/downward quotation of the unit, $\Delta P_{i,b}^{+}$ is the upper adjustment amount of the unit in section b, and $C_{i,b}^{+}$ is the adjustment quotation in section b. $\Delta P_{i,b}^{-}$ (<0) is the down-regulation quantity of the unit $i$ in section b, $C_{i,b}^{-}$ (>0) is the down-regulation quotation of the unit $i$ in section b, and $\Delta P_j^{+}$ and $\Delta P_j^{-}$ (<0) are the up-regulation and down-regulation of unit j. ${\lambda }_j\left(P_j^{rt}\right)$ is the price of the incremental quotation function of unit j at its final output point $P_j^{rt}$, $P_i^{rt}$ and $P_j^{rt}$ are the final output plans of unit i and j, and $P_i^0$ and $P_j^0$ are the previous pre-dispatch plans of unit i and j.

(2)

Constraint conditions

(1) Power balance constraints

In the operation of regional power grids, the dispatch center must first ensure power supply to users within the RPG to ensure system power balance [41,47].

$${\displaystyle \sum _{i=1}^{NI}{P}_i^{rt}}+{\displaystyle \sum _{j=1}^{NJ}{P}_j^{rt}}+{\displaystyle \sum _{g=1}^{NG}{P}_g^0}+{\displaystyle \sum _{m=1}^{NT}{T}_m}=D$$

(18)

where $P_g^0$ is the pre-dispatch plan of unit g, $T_m$ is the planned power of cross-regional tie lines M, NT is the total number of cross-regional tie lines in RPG, and D is the power load of RPG.

(2) Rotation reserve constraints

In the operation of regional power grids, the sum capacity of upward adjustment and downward adjustment of the unit should meet actual operation demand for upward and downward adjustment of the rotary standby [42,43,44].

$${\displaystyle \sum _{i=1}^{NI}\min }\left\{\Delta P_i^U,P_i^{\max }-P_i^{rt}\right\}+{\displaystyle \sum _{j=1}^{NJ}\min }\left\{\Delta P_j^U,P_j^{\max }-P_j^{rt}\right\}+{\displaystyle \sum _{g=1}^{NG}\min }\left\{\Delta P_g^U,P_g^{\max }-P_g^0\right\}\ge \Delta S{R}^U$$

(19)

$${\displaystyle \sum _{i=1}^{NI}\min }\left\{\Delta P_i^D,P_i^{rt}-P_i^{\min }\right\}+{\displaystyle \sum _{j=1}^{NJ}\min }\left\{\Delta P_j^D,P_j^{rt}-P_j^{\min }\right\}+{\displaystyle \sum _{g=1}^{NG}\min }\left\{\Delta P_g^D,P_g^0-P_g^{\min }\right\}\ge \Delta S{R}^D$$

(20)

$${\displaystyle \sum _{i\in {\Omega }_z}\min }\left\{\Delta P_i^U,P_i^{\max }-P_i^{rt}\right\}+{\displaystyle \sum _{j\in {\Omega }_z}\min }\left\{\Delta P_j^U,P_j^{\max }-P_j^{rt}\right\}+{\displaystyle \sum _{g\in {\Omega }_z}\min }\left\{\Delta P_g^U,P_g^{\max }-P_g^0\right\}\ge \Delta S{R}_z^U$$

(21)

$${\displaystyle \sum _{i\in {\Omega }_z}\min }\left\{\Delta P_i^D,P_i^{rt}-P_i^{\min }\right\}+{\displaystyle \sum _{j\in {\Omega }_z}\min }\left\{\Delta P_j^D,P_j^{rt}-P_j^{\min }\right\}+{\displaystyle \sum _{g\in {\Omega }_z}\min }\left\{\Delta P_g^D,P_g^0-P_g^{\min }\right\}\ge \Delta S{R}_{\mathrm{z}}^D$$

(22)

where $\Delta P_i^U$, $\Delta P_j^U$, and $\Delta P_g^U$ are the maximum upward climbing rate of the unit, $\Delta P_i^D$, $\Delta P_j^D$, and $\Delta P_g^D$ are the maximum downward climbing rate of the unit, and $P_i^{\min }$, $P_j^{\min }$, and $P_g^{\min }$ are the minimum technical output of the unit. $P_i^{\max }$, $P_j^{\max }$ and $P_g^{\max }$ are the maximum technical output of the unit, $\Delta S{R}^U$ and $\Delta S{R}^D$ are the backup requirements for regional and province rotation, respectively, and $\Delta S{R}^D$ and $\Delta S{R}_z^D$ are the backup requirements for the downward rotation of the region and province, respectively.

(3) Unit output power constraints

The real-time operation power of the unit should be within its maximum/minimum output power range.

$$P_i^{\min }\le P_i^{rt}\le P_i^{\max }$$

(23)

$$P_{i,b}^{\min }\le \Delta P_{i,b}^{+}\le P_{i,b}^{\max }$$

(24)

$$P_{i,b}^{\min }\le -\Delta P_{i,b}^{-}\le P_{i,b}^{\max }$$

(25)

$$P_j^{\min }\le P_j^{rt}\le P_j^{\max }$$

(26)

(4) Unit climbing constraints

When the unit climbs uphill or downhill, it should meet the climbing rate requirements.

$$P_i^{rt}-P_i^0\le U{R}_i$$

(27)

$$P_i^0-P_i^{rt}\le D{R}_i$$

(28)

$$P_j^{rt}-P_j^0\le U{R}_j$$

(29)

$$P_j^0-P_j^{rt}\le D{R}_j$$

(30)

where $U{R}_i$ and $D{R}_i$ are the uphill and downhill climbing abilities of unit i, respectively, and $U{R}_j$ and $D{R}_j$ are the uphill and downhill climbing abilities of unit j.

(5) RPG lines’ power flow constraints

The line transmission capacity of RPG is limited, and when the power flow exceeds the rated capacity of the transmission line, it may lead to overload operation of the RPG [45,46].

$$-F_l^{\max }\le {\displaystyle \sum _{i=1}^{NI}{G}_{l-i}{P}_i^{rt}}+{\displaystyle \sum _{j=1}^{NJ}{G}_{l-j}{P}_j^{rt}}+{\displaystyle \sum _{g=1}^{NG}{G}_{l-g}{P}_g^0}+{\displaystyle \sum _{m=1}^{NT}{G}_{l-m}{T}_m}-{\displaystyle \sum _{k=1}^{NK}{G}_{l-k}{D}_k}\le F_l^{\max }$$

(31)

where $F_l^{\max }$ is the power flow transmission limit of the RPG lines, $G_{l-i}$, $G_{l-j}$, and $G_{l-g}$ are the distribution factor of generator output power transfer from the node where the unit is located to line l, and $G_{l-m}$ is the distribution factor of generator output power transfer from the node where tie line m is located to line l. $NK$ is the number of system load nodes, $G_{l-k}$ is the distribution factor of generator output power transfer from node k to line l, and $D_k$ is the bus load of the node.

4.2.2. Optimization Model of Technical Program 3-2

(1) Objective function

In Program 3-2, the dispatching center of RPG conducts safety checks on RPG based on the pre-dispatch plans of the provincial power grid and implements congestion management through market coupling re-clearing. The model optimizes and re-clearing with the objective function of minimizing power generation cost. It does not change the unit combination results of pre-dispatch plans of the provincial power grid but only changes the output of units quoted close to the marginal electricity price in each provincial power grid.

$$\min \hspace{1em}\hspace{1em}{\displaystyle \sum _{i=1}^N{C}_i\left(P_i\right)}$$

(32)

$$C_i\left(P_i\right)={\displaystyle \sum _{b=1}^{NB}{C}_{i,b}{P}_{i,b}}$$

(33)

$$P_i={\displaystyle \sum _{b=1}^{NB}{P}_{i,b}}$$

(34)

where N is the total number of startup units, $P_i$ is the output value of unit i, and $C_i\left(P_i\right)$ is the power generation cost of unit i, which is determined by the alternative supply curve. NB is the total number of units quotations, and $C_{i,b}$ and $P_{i,b}$ are the quotation and bid-winning output of unit i in section b, respectively.

(2) Constraint conditions

In Program 3-2, the constraint conditions of power balance constraints, rotation reserve constraints, unit climbing constraints, and RPG lines power flow constraints are consistent with Program 1 and will not be repeated here. In addition, the special unit output power constraints need to be emphasized. Namely, the real-time operation power of the unit should be within its maximum/minimum output power range.

$$P_{i,b}^{\min }\le P_{i,b}\le P_{i,b}^{\max }$$

(35)

$$P_i^{\min }\le P_i\le P_i^{\max }$$

(36)

where $P_{i,b}^{\min }$ and $P_{i,b}^{\max }$ are the upper and lower bounds of section b of the alternative supply curve of unit i, respectively.

4.2.3. Model Improvement of Program 3 Considering Minimizing Change of Tie Line Plan

During the operation of RPG, the tie-line gateway (TLG) plan is an important means to ensure the safe and stable operation of RPG and realize the optimal allocation of power resources [48,49]. TLG plan is a power exchange and scheduling plan made according to the power supply and demand, power grid operation, and cross-regional and inter-provincial power transactions. In order to balance the management of the TLG plan and the adjustment of the pre-dispatch plans of the provincial power grid and achieve the minimization of TLG changes and optimal scheduling, a penalty function for the imbalance adjustment within the provincial power grid is added to the model of Program 3.

For Program 3-1, the objective function of the re-adjustment model is improved as follows:

$$\begin{array}{l}\min \hspace{1em}\hspace{1em}{\displaystyle \sum _{i=1}^{NI}\left[{\displaystyle \sum _{b=1}^{NB}\left(\Delta P_{i,b}^{+}{C}_{i,b}^{+}+\Delta P_{i,b}^{-}{C}_{i,b}^{-}\right)}\right]}+{\displaystyle \sum _{j=1}^{NJ}\left[\Delta P_j^{+}{\lambda }_j\left(P_j^{rt}\right)+\Delta P_j^{-}{\lambda }_j\left(P_j^{rt}\right)\right]}\\ \hspace{1em}\hspace{1em}\hspace{1em}+M{\displaystyle \sum _{z=1}^{N_{ZI}}\left|{\displaystyle \sum _{i\in z}\left({\displaystyle \sum _{b=1}^{NB}\Delta P_{i,b}^{+}}+{\displaystyle \sum _{b=1}^{NB}\Delta P_{i,b}^{-}}\right)}\right|}+M{\displaystyle \sum _{z=1}^{N_{ZJ}}\left|{\displaystyle \sum _{j\in z}\left(\Delta P_j^{+}+\Delta P_j^{-}\right)}\right|}\end{array}$$

(37)

where M is the penalty coefficient, $N_{ZI}$ is the total number of provinces in the RPG that participate in the RPG congestion management by lowering the quotation above, and $N_{ZJ}$ is the total number of provinces in the RPG that participate in the RPG congestion management by increasing the quotation.

For Program 3-2, the objective function of the market coupling re-clearing model is improved as follows:

$$\min \hspace{1em}{\displaystyle \sum _{i=1}^N{C}_i\left(P_i\right)}+M{\displaystyle \sum _{z=1}^{N_Z}\left|{\displaystyle \sum _{i\in z}\left(P_i- P_i^0\right)}\right|}$$

(38)

where $N_Z$ is the total number of provinces in the RPG.

4.3. Model Solving Method

4.3.1. Transformation of Operation Optimization Model Based on MDP

In the congestion management model established in this paper, pre-dispatch plans of the provincial power grid and the TLG plan of RPG are the basic parameters, and the variable of unit output is optimized and adjusted. In the operation optimization models for various programs, complex variables of power flow constraints exist, along with multiple time and quantity constraints. This complexity demands significant calculation resources, making traditional mathematical solutions slow. Currently, energy system electricity market clearing often combines optimization or heuristic algorithms. While optimization algorithms are efficient, they struggle with local optima in non-linear, non-convex, or discontinuous problems. Heuristic algorithms can find optimal solutions or Pareto frontiers under certain conditions but have limitations, including long calculation times and poor generalization.

Reinforcement learning (RL), based on Markov decision processes (MDP), is a machine learning method with strong autonomous learning and adaptability. RL makes sequential decisions in unknown environments, adjusting strategies in real time through online learning from past experiences. It does not require knowledge of system uncertainty distributions, making it a potential solution for optimization problems with uncertainties. RL has been introduced into the electricity market for clearing optimization. Therefore, operation optimization models are transformed into discrete finite horizon MDP, as they represent decision-making in stochastic environments. The MDP process includes states, actions, rewards, and discount rates. Rewards and actions depend only on current energy information, not historical data. Thus, this problem can be modeled as a finite MDP with security-constrained unit commitment as the agent [50,51]. Figure 6 shows the agent–environment interactive mode.

For the operation optimization problem of different programs, its agent is the dispatch center of RPG, and the environment is the provincial power grid with different ESM under various safety operation constraints. Here, the action space $A_{MDP}$, the state space $S_{MDP}$, and the reward function $R_{MDP}$ are modeled once [50,51,52,53].

(1) Action space

For the operation optimization problem considering congestion management, the action represents the unit output power and power of cross-regional tie-lines in different provincial power grids, while the action space is the decision-making space that all generating units can make.

$$A_{MDP}=\left\{P_i,T_m\right\}$$

(39)

where $A_{MDP}$ is the action space, which is determined by the adjustment strategy for pre-dispatch plans in different programs.

(2) State space

For the operation optimization problem considering congestion management, the state space is the RPG lines power flow state of different programs.

$$S_{MDP}=\left\{formula(14),formula(31)\right\}$$

(40)

(3) Reward function

The rationality of reward function setting has a significant influence on the action direction of the agent and the convergence of the algorithm. In this paper, on the basis of the objective function of different programs, considering the operating conditions, the reward function is constructed.

$$R_{MDP}=\left\{formula(5),formula(37),formula(38)\right\}$$

(41)

4.3.2. Solving Algorithm Based on Q-Learning

Q-learning is a widely used model-free reinforcement learning algorithm that is based on time series differences, with simple principles and few parameters. Therefore, this paper adopts the Q-learning algorithm to solve the MDP model and acquire an adjustment strategy for pre-dispatch plans in different programs. The Q-learning algorithm is expressed as follows [50,51,52,53]:

(1)

Initialize the value function $Q(S_{MDP},A_{MDP})$ and set the attenuation factor $\gamma$, learning rate $\mathsf{\theta }$, and exploration rate ε.

(2)

Algorithm iteration until $Q(S_{MDP},A_{MDP})$ converges.

(1) Initialize $S_{MDP}$ as the first state of the current iteration.

(2) Use the ε-greed method to select the action $A_{MDP}$ in the current state $S_{MDP}$ to obtain the instant reward $R_{MDP}$.

(3) Update the Q value $Q(S_{MDP},A_{MDP})$, and the Q value gradually approaches the true value through iterative updating, and the updating formula is based on the Bellman equation:

$$\begin{array}{c}Q(S_{MDP},A_{MDP})\leftarrow Q(S_{MDP},A_{MDP})+\theta [R_{MDP}(S_{MDP},A_{MDP})\hfill \\ \hfill +\gamma \max Q(S_{MDP}^{*},A_{MDP}^{*})-Q(S_{MDP},A_{MDP})]\end{array}$$

(42)

where $\theta$ is the learning rate, essentially the step size for updates, indicating to what extent newly acquired information overrides old information. $\gamma$ is the discount factor, ranging from 0 to 1, which characterizes the impact of immediate versus future rewards. Specifically, a larger $\gamma$ means the agent is more influenced by future rewards, but this also increases the difficulty of learning. A smaller $\gamma$ reduces the learning difficulty but causes the agent to focus more on immediate rewards, potentially leading to local optima. Therefore, the principle for selecting $\gamma$ is to make it as large as possible, provided that the algorithm can still converge.

(4) Update the current state to $S_{MDP}$.

(5) If $S_{MDP}$ is the termination state, the iteration of the current round is completed; otherwise, proceed to (2).

The termination condition is that the difference between the current Q value and the previous Q value is less than a minimal positive number $\delta$. At this point, the Q value converges to the maximum value. The value of $\delta$ depends on the algorithm design. The operation optimization model based on MDP is to find an optimal adjustment strategy for pre-dispatch plans to minimize the value function, as defined in (5), (37), and (38). The optimal adjustment strategy v of the agent can be obtained as follows:

$$v=\arg \max Q(S_{MDP},A_{MDP})$$

(43)

5. Case Study

5.1. Basic Parameters

In this paper, an IEEE-39 node test system is simulated as an example to verify the effectiveness of the three proposed programs for safety check and congestion management. The test RPG is established based on the IEEE 39-bus system, which comprises 39 buses, 46 transmission lines, and 10 generating units. To closely align with the actual situation in the test RPG, the system is divided into three provinces, corresponding to a province with a fixed power generation plan (namely, Without ESM), a province with a centralized ESM, and a province with decentralized ESM. Figure 7 shows the power grid structure of the test system.

Table 5. Installed capacity and load of the test system.

Provinces	Market Model	Installed Capacity (MW)	Load (MW)
Province A	Without ESM	2469	1613.5
Province B	Centralized ESM	2471	2801.63
Province C	Decentralized ESM	2427	1839.1

shows the installed capacity and load of the test system.

In order to verify the advantages of different programs in safety check and congestion management, this section designs the pre-dispatch plans of the provincial power grid and the TLG plan of RPG as the basic parameters.

Table 6. Pre-dispatch plans of the provincial power grid for power generation.

Provinces	Province A			Province B			Province C
Unit number	G1	G8	G9	G2	G3	G10	G4	G5	G6	G7
Out power/MW	886	480.5	736.9	196.6	725	1100	652	210.1	687	580

shows the pre-dispatch plans of the provincial power grid, and Figure 6 shows the TLG plan of RPG.

During the planning process, province A without ESM develops a pre-dispatch plan according to the principles of fairness, openness, and justice. The provinces with ESM, like provinces B and C, take into account market transactions to develop pre-dispatch plans in an economically efficient manner.

Based on the power supply and demand situation in each province, the TLG plan of RPG is formulated. Province A supplies 510 MW of electricity to province B, province C supplies 20 MW of electricity to province A, and province C supplies 270 MW of electricity to province B. Figure 8 shows the power grid interface operation plan of tie line in RPG.

The dispatching center of RPG conducted a safety check for the pre-dispatch plans of the provincial power grid. The results showed that the interconnection line L25 (bus15–bus16) between provinces B and C exceeded its limit; the actual power flow on line L25 was 309.4 MW, which is greater than the line limit of 300 MW.

Based on the security check results and congestion management requirements designed in Section 5.1, the simulation of operation optimization and congestion management based on three established programs is executed, and the congestion management results are shown in Section 5.2. Here, it should be noted that although the models established in programs 1~3 are different, the forms of action, state, and reward function related to MDP are the same. Therefore, three congestion management models are transformed into the MDP model and solved by the Q-learning algorithm. In this paper, the learning rate $\theta$ and discount factor $\gamma$ for the Q-learning algorithm are set to 0.01 and 0.8 [54], the number of iterations is set to 5000, and all the simulations are completed by a PC with an Intel Core i9-13900K CPU @ 3.00 GHZ with 32.00 GB RAM, and the optimal problems are solved using MATLAB software R2024b. The boundary conditions for the case study include the following three aspects.

(1) The simulation scenario design matches the actual situation of RPG in China. The foregoing content in Section 3 has already provided a detailed overview of the complex market environment, coordination processes between RPG and the provincial power grid, and the timing and organizational processes for safety checks.

(2) The simulation scenario is designed based on the safety check and congestion management mechanism for the RPG in China. The detailed design of the participating entities, unit bidding, optimization models, and other aspects of this simulation should be consistent with the mechanism framework proposed in Section 3. Reasonable simplifications or equivalents can be made for the simulation scenario without affecting the simulation results.

(3) The uncertainty factors and unexpected situations are temporarily not considered in scenario design. To highlight the objectives and primary tasks of the case study and avoid the influence of irrelevant factors, the simulation is conducted under a deterministic typical system operation mode. Uncertainty factors such as output fluctuations in renewable energy, deviations in load forecasting, and anticipated power grid incidents are temporarily not considered.

5.2. Simulation Results of Congestion Management

5.2.1. Congestion Management Result for Program 1

Figure 9 shows the congestion management results for Program 1, and

Table A1. Adjustment to generation plan results for each unit in Program 1 (MW).

Unit Number		Province A			Province B			Province C
		G1	G8	G9	G2	G3	G10	G4	G5	G6	G7
Initial output		886.0	480.5	736.9	196.6	725	1100	652	210.1	687	580
First adjustment	Adjusted output	886.0	480.5	736.9	211.6	725	1100	652	195.1	687	580
	Unit output adjustment	0	0	0	15	0	0	0	−15	0	0
Second adjustment	Adjusted output	886.04	480.5	736.9	226.6	725	1100	652	180.1	687	580
	Unit output adjustment	0	0	0	30	0	0	0	−30	0	0
Unit output adjustment in each province		0			30			−30

in the Appendix A shows the adjustment to generation plan results for each unit in Program 1.

In Program 1, based on the overload situation of Line L25, provinces B and C revise their market boundaries, adjust pre-dispatch plans through re-clearing, and conduct a new safety check of the RPG until the system operation requirements are met. After two adjustments and a safety check, the overload situation on Line L25 is ultimately eliminated. Throughout the process, only minor adjustments were made to the output of unit G2 in province B and unit G5 in province C.

5.2.2. Congestion Management Result for Program 2

Figure 10 shows the congestion management results for Program 1, and

Table A2. Adjustment to generation plan results for each unit in Program 2 (MW).

Unit Number	Province A			Province B			Province C
	G1	G8	G9	G2	G3	G10	G4	G5	G6	G7
Initial output	886	480.5	736.9	196.6	725	1100	652	210.1	687	580
Adjusted output	886	480.5	715	221.2	725	1100	649.4	210.1	687	580
Unit output adjustment	0	0	−22	24.5	0	0	−2.5	0	0	0
Unit output adjustment in the province	−22			24.5			−2.5

in the Appendix A shows the adjustment to generation plan results for each unit in Program 1.

In Program 2, the dispatching center of RPG conducts congestion management with the objective function of minimizing adjustments based on the sensitivity of unit output to line power flow. The line overload issue is eliminated in Program 2 with just one adjustment by increasing the output of unit G2 in province B, decreasing the output of unit G9 in province A and unit G4 in province C, and the total adjustment is 24.5 MW.

5.2.3. Congestion Management Result for Program 3

On the basis of simultaneously considering congestion management and minimizing the TLG plan, two strategies in Program 3 were simulated in this section.

(1) Congestion management results for Program 3-1

In Program 3-1, the dispatching center of RPG conducts RPG congestion management through re-adjustments based on market participant bids. Here, three cases are designed to validate and simulate the model in Program 3-1.

Case 1: Without considering minimizing TLG plan changes for all units in province A participating in adjustment;

Case 2: Considering minimizing TLG plan changes for all units in province A participating in adjustment;

Case 3: Considering minimizing TLG plan changes for only unit G1 in province A participating in adjustment

Figure 11 shows the congestion management results for Program 3-1, and

Table A3. Adjustment to generation plan results for each unit in Program 3-1 (MW).

Unit Number		Province A			Province B			Province C
		G1	G8	G9	G2	G3	G10	G4	G5	G6	G7
Initial output		886.0	480.5	736.9	196.6	725	1100	652	210.1	687	580
Case 1	Adjusted output	1011.0	87.5	865.0	646.0	725.0	1100.0	652.0	0.0	687.0	480.8
	Unit output adjustment	124.9	−393	128.1	449.4	0	0	0	−210.1	0	−99.2
	Unit output adjustment in each province	−140.0			449.4			−309.3
Case 2	Adjusted output	1040.0	411.9	651.6	196.9	725.0	1100.0	652.0	209.9	687.0	580.0
	Unit output adjustment	154.0	−68.6	−85.3	0	0	0	0	0	0	0
	Unit output adjustment in each province	0			0			0
Case 3	Adjusted output	851.4	480.5	736.9	231.2	725.0	1100.0	652.0	210.1	687.0	580.0
	Unit output adjustment	−34.6	0	0	34.6	0	0	0	0	0	0
	Unit output adjustment in each province	−34.6			34.6			0

in the Appendix A shows the adjustments to generation plans for each unit in Program 3-1.

It can be observed that when minimizing TLG plan changes is not considered, the power adjustment for units in the entire RPG is the largest, and the final power flow on Line L25 is much lower than its limit. The reasons for the above results include two aspects. On the one hand, all units in the province without ESM participate in adjustment, which will inevitably result in the replacement of generation rights within the province, where low-cost units replace high-cost units for power generation. This has a significant impact on the provincial power generation plan. On the other hand, when minimizing TLG plan changes is not considered, it will also lead to the replacement of generation rights between provinces, where low-cost units within a province replace high-cost units in other provinces for power generation, which ultimately leads to substantial adjustments to the TLG plan of RPG and pre-dispatch plan of the provincial power grid.

(2) Congestion management results for Program 3-2

but only changes the output of units quoted close to the marginal electricity price in each provincial power grid.

In Program 3-2, the dispatching center of RPG conducts RPG congestion management through market coupling and re-clearing. The optimization strategy is to adjust the output of units quoted to be close to the marginal electricity price. Here, four cases are designed to validate and simulate the model in Program 3-2.

Case 1: Without considering minimizing TLG plan changes for all the units participating in adjustment;

Case 2: Considering minimizing TLG plan changes for all units participating in adjustment;

Case 3: Considering minimizing TLG plan changes for only the units quoted close to the marginal electricity price participating in adjustment (unit G1 in province A; unit G2 in province B; unit G5 in province C);

Case 4: On the basis of Case 3, unit G3 in province B also participated in the adjustment.

Figure 12 shows the congestion management results for Program 3-2, and

Table A4. Adjustment to generation plan results for each unit in Program 3-2 (MW).

Unit Number		Province A			Province B			Province C
		G1	G8	G9	G2	G3	G10	G4	G5	G6	G7
Initial output		886.0	480.5	736.9	196.6	725	1100	652	210.1	687	580
Case 1	Adjusted output	967.0	32.3	865.0	646.0	725.0	1100.0	652.0	0.0	687.0	580.0
	Unit output adjustment	80.9	−448.2	128.1	449.3	0	0	0	−210.1	0	0
	Unit output adjustment in each province	−293.3			449.3			−210.1
Case 2	Adjusted output	1040.0	411.9	651.6	196.9	725.0	1100.0	652.0	209.9	687.0	580.0
	Unit output adjustment	154.01	−68.6	−85.3	0	0	0	0	0	0	0
	Unit output adjustment in each province	0			0			0
Case 3	Adjusted output	851.4	480.5	736.9	231.2	725.0	1100.0	652.0	210.1	687.0	580.0
	Unit output adjustment	−34.6	0	0	34.6	0	0	0	0	0	0
	Unit output adjustment in each province	−34.6			34.6			0
Case 4	Adjusted output	851.4	480.5	736.9	231.2	725.0	1100.0	652.0	210.1	687.0	580.0
	Unit output adjustment	−34.6	0	0	34.6	0	0	0	0	0	0
	Unit output adjustment in each province	−34.6			34.6			0

in the Appendix A shows the adjustments to generation plans for each unit in Program 3-2.

Similar to Program 3-1, it can be observed that when minimizing TLG plan changes is not considered, the power adjustment for units in the entire RPG is the largest, and the final power flow on Line L25 is much lower than its limit. When considering minimizing TLG plan changes, the results of the congestion management in RPG are significantly influenced by the scope of units participating in the adjustment. On the one hand, if all units in province A participate in the adjustment, there will be a replacement of generation rights within the province, which will have a significant impact on the provincial power generation plan. On the other hand, increasing the selection range of units quoted close to the marginal electricity price in the provinces with ESM has little impact on the final adjustment results.

5.3. Results and Discussion

Table 7. Iterations and solution time in different programs and cases.

Programs	Program 1	Program 2	Program 3-1	Program 3-2
Total iterations	5000	5000	5000	5000
Iterations in convergence	2956	2915	[3891, 3911, 3903]	[3615, 3805, 3711, 3708]
Time taken for convergence	95 s	89 s	[119 s, 126 s, 122 s]	[103 s, 116 s, 112 s, 109 s]

shows the iterations and solution times in different programs and cases. As can be seen from Table 7, under the same number of iterations, running environment, and algorithm parameter settings, the convergence iteration count and solution time of the algorithm vary significantly across different programs and cases, closely correlating with the complexity of each program’s model and the number of units involved in regulation within each case.

Besides, according to the analysis results in Section 5.2, it can be concluded that not considering minimizing TLG plan changes during the congestion management process in Program 3 will result in significant adjustments to the TLG plan. Figure 12 and Figure 13 show the TLG plan with or without considering minimizing TLG plan changes in Program 3.

As shown in Figure 13, in the original TLG plan of RPG, the total electricity output from province A to other provinces is 490 MW, the total electricity imported from other provinces in province B is 780 MW, and the total electricity output from province C to other provinces is 290 MW. After adjusting through Program 3-1, the total electricity output from province A to other provinces was reduced to 350 MW, the total electricity imported from other provinces in province B was reduced to 330 MW, and province C shifted from transmitting electricity to other provinces to importing electricity from other provinces, with an input of 20 MW. Moreover, after adjusting through Program 3-2, the total electricity output from province A to other provinces was reduced to 250 MW, the total electricity imported from other provinces in province B was reduced to 330 MW, and the total electricity output from province C to other provinces reduced to 80 MW. As shown in Figure 14, if we consider minimizing the TLG plan changes and involving all units in adjustment, the TLG plan changes for various programs are relatively small.

6. Conclusions

In this paper, a framework of security check and congestion management mechanism for RPG is designed, and the operation optimization method considering congestion management is established for three different security check and congestion management programs. The following conclusions are drawn from the simulation results:

(1) The mechanism and simulation analysis results show that Program 3, based on market mechanisms, has theoretical and practical advantages over Program 1 (based on multiple adjustments) and Program 2 (based on dispatch plans) for congestion management. The simulation results show that under the same line congestion situation, Program 1 requires two adjustments to relieve the line congestion, while Program 2 and Program 3 can solve the problem with just one optimization adjustment. Moreover, the congestion management effect of Program 3 is significantly better than that of Program 2, which can be seen from the changes in line power before and after optimization.

(2) Whether to consider adjusting the TLG plan during the congestion management process has a great impact on the congestion management results. The simulation results show that if minimizing the change in TLG is not taken into account, there will be a significant adjustment to the TLG plan. When taking into account TLG management, minimizing the change in TLG, and with all units participating in the regulation, the adjustments to the TLG plan under various schemes are relatively minor. If the TLG plan constraints are not considered in RPG security check and congestion management modeling, a large number of power generation rights will be replaced among the provinces in RPG due to power generation cost, which will lead to confusion in RPG scheduling results, break the original power supply and demand balance of the provincial power grid, and the adjustment of generation plans of the provincial power grid will be greater.

The research primarily focuses on designing feasible RPG security check mechanisms and tailored congestion management models for China’s unique power dispatch system and complex electricity market. It verifies the feasibility and scientific validity of the proposed program and model. However, current studies have not thoroughly considered the robustness of the models or their practical engineering applications, such as actual power losses and the uncertainty/volatility of renewable energy output. Future research will expand the existing model, incorporating a robust optimization approach that accounts for uncertainty and power loss costs. This will be validated through real RPG to enhance the practicability of the proposed mechanisms.