Network Dynamic Spatiotemporal Dispatch Based on Multi-Head Graph Attention Reinforcement Learning and Balanced Responsibility

Abstract

The rapid integration of distributed renewable energy and flexible loads significantly intensifies supply and demand uncertainty in active distribution networks (ADNs), threatening economic and secure grid operations. Existing deep reinforcement learning (DRL) dispatch methods fail to extract spatial features properly, leading to a local optimal solution. To address these limitations, this paper proposes a state-adaptive topology-aware continuous-dispatch framework via multi-head graph attention network and deep deterministic policy gradient (GAT-DDPG). A multi-head graph attention network is embedded within a centralized Actor–Critic training paradigm to adaptively update spatial message-passing weights based on operational states. Extensive simulations on a modified IEEE 33-bus ADN over a 125-day unseen test set demonstrate that the proposed framework achieves lower comprehensive operating costs and fewer voltage violations compared with representative DRL-based dispatch baselines. Visualizations of state-dependent attention shifts confirm the model’s capability to track dynamically shifting network vulnerabilities, providing physical interpretability. Based on this, and combined with the optimized dispatch method of balancing responsibility, the ability of different flexible resources to support safe and stable operation and the ideal dispatch results under the temporary reduction in new energy output are further simulated.

1. Introduction

The global transition toward low-carbon power systems has accelerated the integration of distributed renewable energy sources (RESs) and flexible loads, transforming traditional passive networks into multi-source active distribution networks (ADNs) [1,2,3,4]. However, the deep spatiotemporal coupling of highly uncertain RESs and flexible loads significantly intensifies operational volatility, posing severe threats to the economic and secure operation of the grid [5]. Therefore, effectively exploiting the regulation capability of distributed resources and achieving secure, efficient, and flexible scheduling of ADNs have become critical research issues in intelligent distribution systems.

To address the uncertainty associated with renewable generation and load demand in ADNs, conventional operation research-based model-driven optimization methods, including stochastic programming and robust optimization, have been widely adopted for scheduling decision-making [6]. These methods characterize uncertain operating scenarios through mathematical optimization formulations and have been further applied to flexibility reserve capability assessment [7] and multi-time-scale scheduling optimization [8,9]. However, with the large-scale integration of distributed renewable energy, energy-storage systems, and flexible loads, ADNs exhibit increasingly high-dimensional, multi-agent coupled, and highly time-varying characteristics. Consequently, traditional optimization methods require repeated solutions of complex mathematical models, resulting in considerable computational burdens and difficulty in satisfying the requirements of rapid decision-making [10].

To overcome these limitations, data-driven deep reinforcement learning (DRL) has gradually been applied to ADN scheduling problems. By interacting with the environment, DRL agents can learn optimal control policies without repeatedly solving complex optimization models, thereby improving scheduling efficiency [11]. Among existing DRL methods, the deep Q-network (DQN) integrates deep neural networks with Q-learning to achieve decision-making in complex state spaces [12]. However, its predefined discrete action space limits its applicability to continuous control tasks. Proximal policy optimization (PPO) improves training stability by constraining policy updates [13], but its stochastic policy sampling mechanism may reduce training efficiency in high-dimensional continuous control problems. In contrast, the deep deterministic policy gradient (DDPG) method based on the Actor–Critic framework can handle continuous action spaces by generating resource regulation strategies through the Actor network and evaluating action values through the Critic network. Therefore, DDPG is particularly suitable for coordinated optimization of continuous control variables, such as distributed generators, energy-storage systems, and flexible loads [14]. Nevertheless, existing DRL-based methods generally represent system states as vectorized inputs, which limits their ability to exploit the spatial topology information embedded in nodes, transmission lines, and electrical connectivity relationships of ADNs.

To enhance the spatial perception capability of DRL agents, graph neural networks (GNNs) have recently been introduced into ADN optimization and scheduling problems [15]. Compared with conventional vector-based state representations, graph-based learning methods can effectively extract spatial correlations by incorporating the connectivity relationships among nodes and lines. Specifically, multi-layer perceptron (MLP)-based data-driven methods typically learn nonlinear mappings between system states and control objectives for ADN optimization [16]. However, these approaches mainly rely on vectorized representations and cannot explicitly capture the spatial topology information among network components. In comparison, graph convolutional networks (GCNs) improve spatial feature representation by aggregating information among connected nodes according to ADN topology [17]. Nevertheless, GCN-based methods generally rely on fixed adjacency matrices for information propagation, making it difficult to adaptively adjust the importance of node interactions under different operating conditions. Graph attention networks (GATs) further introduce attention mechanisms to dynamically learn the aggregation weights among different nodes, enabling the model to focus on critical spatial relationships according to changing operating states and improving the adaptability of scheduling strategies in complex scenarios [18].

Based on these advantages, hybrid models integrating GNNs and reinforcement learning have recently attracted increasing attention for ADN optimization decision-making. By extracting spatial topology features through GNNs and generating optimization policies through reinforcement learning, these methods improve the capability of agents to perceive complex network states. Among them, graph convolutional networks and deep deterministic policy gradient (GCN-DDPG) [19] and graph attention network and deep deterministic policy gradient (GAT-DDPG) [20] employ graph neural networks as spatial feature extractors and combine them with DDPG for continuous decision-making, effectively integrating network topology information with resource regulation capabilities and achieving topology-aware optimization of ADNs. For coordinated resource scheduling in ADNs, DDPG-based methods have been applied to the joint optimization of distributed generators, energy-storage systems, and flexible loads, achieving economic scheduling under complex operating conditions [14]. Furthermore, Xing et al. proposed a graph reinforcement learning-based scheduling method for ADNs, which integrates the spatial feature extraction capability of GAT with the continuous control capability of DDPG, enabling real-time optimal scheduling considering network topology information [15], and the description is listed in

Table 1. The comparison and description of method.

Method	Ref.	Description
Stochastic optimization	[6,8]	Models uncertainty with scenarios
Robust optimization	[7,9]	Handles worst-case uncertainty
Deep Q-network (DNQ)	[12]	Discrete decision optimization
Proximal policy optimization (PPO)	[13]	Stable policy optimization
Deep deterministic policy gradient (DDPG)	[14]	Continuous control optimization
Graph convolutional networks and deep deterministic policy gradient (GCN-DDPG)	[19]	Topology-aware decision making
Graph attention network and deep deterministic policy gradient (GAT-DDPG)	[15,20]	Attention-based topology-aware scheduling

.

However, deviations inevitably exist between economic scheduling results and actual system outputs due to the temporal coupling characteristics and uncertainty of real-world operation. For instance, in the Shandong real-time electricity market, generation schedules are updated in a rolling manner for the next two hours based on real-time operating conditions [21]. Nevertheless, deviations between scheduled and actual outputs are unavoidable due to renewable energy forecasting errors, load fluctuations, and operational constraints of power system components. To maintain real-time power balance, the Shandong electricity market evaluates actual execution deviations through deviation responsibility identification and assessment mechanisms, while fast balancing capability is provided through ancillary services such as frequency regulation and ramping services.

Similarly, in Germany, balancing groups allow market participants to aggregate resources from geographically distributed locations through digital platforms, requiring dynamic balancing between electricity supply and demand to mitigate real-time deviations [22]. The imbalance of Balance Responsible Parties (BRPs), defined as the deviation between the scheduled electricity position and the actual power injection or withdrawal, is settled by transmission system operators (TSOs), resulting in imbalance prices. However, imbalance price calculation mechanisms vary significantly among European countries [23]. Overall, although electricity markets differ across regions, their common objective is to maintain system balance and ensure secure and stable operation. Inspired by these market mechanisms, balancing responsibility can be understood as the obligation of entities to compensate for deviations between scheduled and actual power exchanges, thereby maintaining real-time power balance. Therefore, based on economic scheduling results, balancing responsibility can be allocated among different entities, enabling flexible resources to participate in corrective adjustments and further improving scheduling performance. In this study, the proposed balancing-responsibility mechanism provides an operator-oriented coordinated redispatch strategy by quantifying the contribution capability of different flexible resources and guiding their participation in renewable deviation compensation.

To bridge these research gaps, a state-adaptive topology-aware continuous-dispatch framework using multi-head GAT-DDPG is proposed. Specifically, a multi-head GAT feature extractor is embedded within a centralized Actor–Critic training paradigm to dynamically update spatial weights based on operational states, providing a more flexible state-dependent spatial aggregation mechanism compared with traditional GCNs. This architecture is supported by a customized decoupled state space that separates systemic global parameters from localized nodal features to strictly safeguard temporal energy boundaries. Extensive validations on an unseen test set demonstrate that the proposed framework achieves optimal economic benefits while significantly minimizing voltage violations compared to baselines. Furthermore, combined with the optimized dispatch method of balancing responsibility, the ability of different flexible resources to support safe and stable operation and the ideal dispatch results under the temporary reduction in new energy output are further simulated.

2. A Combined Consideration of Balancing Responsibility and Economic Allocation

2.1. Considering the Impact Mechanism of Balancing Responsibility and Economic Dispatch

China’s power system adopts a “unified dispatch and hierarchical management” model, with national, grid, provincial, regional, and county dispatch centers, respectively.

(1)

National dispatch center: The highest-level national dispatch center mainly formulates national power trading plans, coordinates inter-regional power transmission, and responds to major accidents;

(2)

Grid dispatch center: The grid dispatch center is responsible for coordinating inter-provincial power trading within its region;

(3)

Provincial dispatch center: The provincial dispatch center needs to balance the power generation and consumption plans within the province and conduct economic operation analysis of the power flow situation within the province;

(4)

Regional dispatch center: The regional dispatch center mainly optimizes the operation of the 110 kV/35 kV distribution network.

(5)

County dispatch center: The county dispatch center mainly monitors the operation of rural power grids at 10 kV and below and directs switching operations to ensure continuous power supply to users.

The traditional safe and stable operation of the power system is accomplished by the provincial dispatch center, which mainly performs verification based on Security Constrained Unit Commitment (SCUC) and Security Constrained Economic Dispatch (SCED) and regulates centralized new energy wind and solar and thermal power resources to ensure supply and demand balance, with thermal power acting as the supporting source. However, as the market accelerates, power generation resources are no longer limited to traditional power sources, giving rise to distributed photovoltaics with dispersed layouts and large data volumes, which presents challenges in detection and control. Furthermore, new types of loads with diverse load characteristics are also increasing on the load side Therefore, if the adjustable potential of load-side resources can be rationally utilized within a limited topological network, and balancing responsibilities can be assumed based on economic dispatching, the balance of the topological region can be guaranteed, thereby promoting the safe and stable operation of the system.

2.2. The Potential and Ability of Differentiated Resources to Assume Active Distribution Network Balancing Responsibility

As a dispatchable distributed generation unit in an active distribution network, micro gas turbines (MTs) have a natural advantage in balancing negative deviations in photovoltaic power generation. MTs use natural gas as fuel, and due to their low mechanical and thermal inertia, they can achieve short start-up times and rapid switching between partial and full loads [24]. This allows them to quickly increase active power to fill power gaps when photovoltaic output drops sharply. Furthermore, unlike intermittent power sources such as wind power, MTs’ regulation capability is independent of weather conditions, maintaining stable output even during cloudy or overcast periods when photovoltaic output is consistently low. They also exhibit strong time-independent characteristics, enabling real-time tracking and compensation for renewable energy fluctuations through optimized dispatch.

Energy-storage systems (ESS), with their millisecond to second-level power response speed and bidirectional regulation capability, are widely considered one of the optimal resources for compensating for uncertainties in new energy output [25]. In power systems, energy-storage systems (ESSs) can compensate for power gaps through additional charging and discharging when photovoltaic and wind power prediction deviations occur, as well as meet the requirements for rapid fluctuations in system power and timeliness [26,27]. Moreover, reasonable energy-storage configuration can not only smooth out intraday photovoltaic fluctuations but also achieve economic benefits through peak-valley arbitrage [28].

Electric vehicle charging stations (EVCS), as flexible resources with both load attributes and regulation potential, provide a unique demand-side balancing responsibility path for active distribution networks. EVCS adjusts the net load of the system by transferring charging power (i.e., “load shedding” or “charging delay”), thereby compensating for photovoltaic output deviation [29]. Furthermore, by fully utilizing the incentive effect of electric vehicle charging prices, it can also give full play to the interaction between vehicles and the grid, thereby encouraging electric vehicle users to shift their charging time from peak to off-peak hours to balance system demand [30].

2.3. Thesis Framework

The Multi-Head GAT-DDPG framework is proposed to overcome the limitations of conventional model-free deep reinforcement learning algorithms in capturing the complex spatial correlations of active distribution networks (ADNs). As shown in the overall framework presented in Figure 1, a decoupled state-space representation is first constructed by separating system-level global features from node-level local features. The global features include time information, electricity prices, and the states of charge (SOCs) of the ESSs, whereas the nodal feature matrix contains active and reactive power injections, device-location indicators, and nodal ESS SOC information. The nodal features, together with the ADN adjacency matrix, are fed into the multi-head graph attention network to extract topology-aware spatial representations, while the global features bypass the graph attention layers. In the graph attention network module of Figure 1, the circles represent graph nodes, and the different colors used in the three attention heads distinguish the corresponding attention-based feature-extraction processes. The resulting spatial features are then concatenated with the global features and provided as inputs to the Actor and Critic networks. In the Actor–Critic module, the green, blue, and orange circles represent neurons in the input, hidden, and output layers, respectively.

During the training phase, the training set is used to learn a continuous-dispatch policy for flexible resources, including MTs, ESSs, and EVCSs, under diverse operating conditions. Based on the comprehensive system state, the Actor generates dispatch actions, including MT power outputs, ESS charging and discharging powers, and EVCS load reduction ratios. After these actions are executed in the simulation environment, power-flow calculations are performed to evaluate the corresponding operating costs, voltage conditions, and constraint violations. The resulting transition samples and rewards are stored in the experience replay buffer and used to update the Actor and Critic networks. After training, the parameters of the GAT-DDPG agent remain fixed, and the test set is used to evaluate the generalization capability of the learned policy under previously unseen operating scenarios.

During testing, the trained agent directly generates dispatch actions according to the state information of each test scenario, thereby obtaining the benchmark economic dispatch results. Building on these results, a balancing-responsibility-based redispatch model is further developed to address resource uncertainties, particularly renewable generation forecast errors. The model fully utilizes the available regulation capabilities of flexible resources and jointly considers economic operating costs, voltage violations, line congestion, balancing-responsibility fulfillment deviations, and responsibility transfers. It is also subject to power-balance constraints, MT output and ramping limits, ESS power and SOC constraints, end-of-day SOC recovery requirements, EVCS flexibility limits, and network security constraints. A differential evolution algorithm is then employed to determine the optimal balancing-responsibility allocation and redispatch scheme, thereby compensating for renewable generation deviations and supporting the secure and stable operation of the ADN.

From the practical implementation perspective, the proposed framework can operate in a day-ahead and real-time coordinated manner. In the day-ahead stage, the trained GAT-DDPG agent provides the baseline economic dispatch strategy based on forecasted renewable generation and load profiles. During real-time operation, actual measurements are collected from smart meters and energy management systems, and the balancing-responsibility redispatch model is activated when deviations occur between scheduled and actual outputs.

In this process, different flexible resources exhibit complementary balancing characteristics and may experience different operational impacts when participating in balancing-responsibility coordination. Specifically, MTs mainly provide fast active power regulation to compensate for renewable generation deviations, which may increase fuel consumption due to additional output adjustments. ESSs participate through bidirectional charging and discharging regulation, where their state of charge (SOC) variations and degradation costs are considered during coordinated dispatch. For EVCSs, balancing responsibility is achieved by adjusting flexible charging demand within predefined limits rather than interrupting basic charging requirements. The corresponding charging adjustment cost is incorporated into the optimization model to represent the potential economic impact on EVCS operators and maintain acceptable charging service quality.

3. Proposed Topology-Aware Deep Reinforcement Learning Framework Considering Balancing Responsibility

3.1. Economic Dispatch Model Optimization

3.1.1. Economic Dispatch Objective Function

The day-ahead dynamic dispatch of ADNs aims to minimize the comprehensive operating cost by coordinating flexible MTs, ESSs, and EVCSs, subject to the physical constraints of secure system operation.

The objective is to minimize the comprehensive economic operating cost C_eco of the ADN while guaranteeing the security of nodal voltages and branch power flows. The objective function is mathematically formulated as follows:

$$\min F=C_{\mathrm{eco}}+{\lambda }_{\mathrm{v}}{P}_{\mathrm{v}}+{\lambda }_{\mathrm{c}}{P}_{\mathrm{c}}$$

(1)

where λ_v and λ_c are the weight factors for voltage and congestion penalties, respectively; and P_v and P_c are the voltage limit and congestion penalties, respectively.

The components of the comprehensive economic operating cost C_eco are as follows:

$$C_{\mathrm{eco}}=C_{\mathrm{grid}}+C_{\mathrm{MT}}+C_{\mathrm{ESS}}+C_{\mathrm{EVCS}}$$

(2)

where: C_grid, C_MT, C_ESS and C_EVCS denote the main grid interaction cost, the MT fuel cost, the ESS degradation cost and the compensation cost for EVCS load curtailment, respectively.

The specific economic components are calculated as follows:

$$C_{\mathrm{grid}}={\displaystyle \sum _{t=1}^T{\lambda }_t^{\mathrm{grid}}\cdot P_t^{\mathrm{grid}}\cdot \Delta t}$$

(3)

$$C_{\mathrm{MT}}={\displaystyle \sum _{t=1}^T{\lambda }_{\mathrm{MT}}\cdot P_t^{\mathrm{MT}}\cdot \Delta t}$$

(4)

$$C_{\mathrm{ESS}}={\displaystyle \sum _{t=1}^T{\lambda }_{\mathrm{ESS}}\cdot \left(\left|P_t^{\mathrm{ch}}\right|+\left|P_t^{\mathrm{dis}}\right|\right)\cdot \Delta t}$$

(5)

$$C_{\mathrm{EVCS}}={\displaystyle \sum _{t=1}^T{\displaystyle \sum _{i\in {\Omega }_{\mathrm{EVCS}}}{\lambda }_i^{\mathrm{EVCS}}}\cdot {\alpha }_{i,t}\cdot P_{i,t}^{\mathrm{EV}\_\max }\cdot \Delta t}$$

(6)

where ${\lambda }_t^{\mathrm{grid}}$ is the time-of-use (TOU) electricity price at time t; λ_MT, λ_ESS, ${\lambda }_i^{\mathrm{EVCS}}$ denote the unit cost coefficients of MT fuel consumption, ESS charging and discharging degradation, and EVCS load curtailment compensation at node i;, respectively; $P_t^{\mathrm{grid}}$, $P_t^{\mathrm{MT}}$, $P_t^{\mathrm{ch}}$, $P_t^{\mathrm{dis}}$, $P_{i,t}^{\mathrm{EV}\_\max }$ denote the power exchanged with the main grid, the output power of the MT, the ESS charging power, the ESS discharging power, and the EVCS charging load at node i and time t, respectively; α_i,t is the curtailment ratio of the EVCS at time t of node i; and Δt denotes the dispatch time interval within the overall time period T.

To promote compliance with power grid security limits and guide the agent towards feasible operational regions, penalties for both nodal voltage violations and branch thermal congestion:

$$P_{\mathrm{v}}={\displaystyle \sum _{t=1}^T{\displaystyle \sum _{i=1}^N{\left(\max \left(0,V_{\min }-V_{i,t}\right)+\max \left(0,V_{i,t}-V_{\max }\right)\right)}^2}}$$

(7)

$$P_{\mathrm{c}}=\sum _{t=1}^T\sum _{l=1}^{N_{\mathrm{line}}}{\left(\max \left(0,L{F}_{l,t}-100\right)\right)}^2$$

(8)

where N is the number of nodes; N_line the number of lines connected to the node; V_i,t is the actual voltage magnitude at node i; [V_min, V_max] is the secure voltage range set to [0.95, 1.05] p.u.; and LF_l,t denotes the actual loading factor of branch l at time t.

3.1.2. ADN Component Models

(1)

MT constraints.

The operation of the MT $P_t^{\mathrm{MT}}$ is subject to output capacity and ramp-rate constraints:

$$P_{\min }^{\mathrm{MT}}\le P_t^{\mathrm{MT}}\le P_{\max }^{\mathrm{MT}}$$

(9)

$$-R_d\le P_t^{\mathrm{MT}}-P_{t-1}^{\mathrm{MT}}\le R_u$$

(10)

where $P_{\min }^{\mathrm{MT}}$ and $P_{\max }^{\mathrm{MT}}$ represent the minimum and maximum power output limits, and R_d and R_u denote the maximum ramp-down and ramp-up rates, respectively.

(2)

ESS constraints.

The charging/discharging state of the ESS should satisfy the dynamic continuity equation of the State of Charge (SOC) and power constraints:

$$SO{C}_t={\displaystyle \frac{{\eta }_{\mathrm{ch}}{P}_t^{\mathrm{ch}}}{E_{\mathrm{cap}}}}+{\displaystyle \frac{P_t^{\mathrm{dis}}}{{\eta }_{\mathrm{dis}}{E}_{\mathrm{cap}}}}+SO{C}_{t-1}$$

(11)

$$SO{C}_{\min }\le SOC(t)\le SO{C}_{\max }$$

(12)

$$\left\{\begin{array}{l}\left|P_t^{\mathrm{ch}}\right|\le P_{\max }^{\mathrm{ch}}\\ \left|P_t^{\mathrm{dis}}\right|\le P_{\max }^{\mathrm{dis}}\end{array}\right.$$

(13)

where η_dis, η_ch denote the ESS discharging and charging efficiency; E_cap denotes the total capacity of the ESS; SOC_max, SOC_min denote the upper and lower limits of the state of charge; and $P_{\max }^{\mathrm{ch}}$, $P_{\max }^{\mathrm{dis}}$ denote the maximum charging and discharging power of the ESS.

(3)

Grid constraints.

$$-P_{\max }^{\mathrm{grid}}\le P_t^{\mathrm{grid}}\le P_{\max }^{\mathrm{grid}}$$

(14)

where $P_{\max }^{\mathrm{grid}}$ denotes the maximum limit of interaction power between the ADN and the upper-level grid.

(4)

Spatiotemporal model of heterogeneous EVCSs.

To capture the spatial heterogeneity and temporal uncertainty of charging demands, the charging load $P_{i,t}^{\mathrm{EV}\_\max }$ of an EVCS at node i is formulated as a spatiotemporal baseline overlaid with stochastic noise:

$$P_{i,t}^{\mathrm{EV}\_\max }=P_{i,t}^{\mathrm{base}}+{\epsilon }_{i,t},{\epsilon }_{i,t}\sim \mathrm{N}\left(0,{\sigma }_{i,t}^2\right)$$

(15)

where $P_{i,t}^{\mathrm{base}}$ represents the baseline load of EVCS at node i; ε_i,t represents the random disturbance term of the charging load at node i during time period t, used to characterize the random fluctuation in actual charging demand relative to the baseline load; and σ_i,t represents the standard deviation of the load fluctuation in EVCS at node i during time period t.

The actual dispatched EVCS power $P_{i,t}^{\mathrm{EVCS}}$ should satisfy:

$$P_{i,t}^{\mathrm{EVCS}}=\left(1-{\alpha }_{i,t}\right)P_{i,t}^{\mathrm{EV}\_\max },0\le {\alpha }_{i,t}\le {\alpha }_{\max }$$

(16)

where α_max represents the maximum allowable shedding margin. This formulation guarantees the satisfaction of basic charging demands while providing operational flexibility.

3.1.3. Markov Decision Process Formulation

The dynamic dispatch problem is formulated as an Markov Decision Process (MDP) defined by the tuple ⟨S,A,R,γ⟩.

(1)

State space.

The global stream ${\mathit{X}}_t^{\mathrm{glb}}$ in the state space captures the global operational conditions and contains two energy-storage systems:

$${\mathit{X}}_t^{\mathrm{glb}}={\left[t,{\lambda }_t^{\mathrm{grid}},SO{C}_{1,t},SO{C}_{2,t}\right]}^{\top }$$

(17)

where $SO{C}_{1,t}$, $SO{C}_{2,t}$ represent the energy-storage state of charge at time t.

The local spatial stream ${\mathit{X}}_t^{\mathrm{node}}$ encodes the topological feature of all N nodes. The 8-dimensional physical attribute vector x_i,t for a specific node i is defined as follows:

$${\mathit{x}}_{i,t}={\left[P_{i,t}^{\mathrm{inj},},Q_{i,t}^{\mathrm{inj},},I_{i,t}^{\mathrm{MT}},I_{i,t}^{\mathrm{ESS}},I_{i,t}^{\mathrm{EV}},I_{i,t}^{\mathrm{PV}},I_{i,t}^{\mathrm{Wind}},SO{C}_{i,t}\right]}^{\top }$$

(18)

where $P_{i,t}^{\mathrm{inj}}$, $Q_{i,t}^{\mathrm{inj}}$, and SOC_i,t represent the active, reactive power injection and the energy-storage state of charge at node i at time t. $I_{i,t}^{(·)}$∈{0, 1} are Bool indicators denoting the topological presence of respective devices.

The comprehensive state space S_t is formulated as a M-dimensional continuous vector, constructed by concatenating the global feature vector ${\mathit{X}}_t^{\mathrm{glb}}$ and the flattened nodal feature matrix ${\mathit{X}}_t^{\mathrm{node}}$, flatten() means flattening the matrix into a one-dimensional vector.

$${\mathit{S}}_t=\left\{\left[{\mathit{X}}_t^{\mathrm{glb}}‖\mathrm{flatten}\left({\mathit{X}}_t^{\mathrm{node}}\right)\right]\in {ℝ}^{\mathrm{M}}\right\}$$

(19)

$$M=\dim ({\mathit{x}}_{i,t})\cdot N+\dim ({\mathit{X}}_t^{\mathrm{glb}})$$

(20)

where dim() represents the spatial dimension.

Furthermore, the SOC information is included in both the global and nodal features. In the global stream, SOC provides the Actor with the overall energy boundary of the ESSs for temporal dispatch decisions. In the nodal stream, SOC is assigned to the corresponding ESS buses, enabling the GAT to distinguish the regulation capability of different ESS nodes under varying network conditions.

(2)

Action space.

To facilitate stable gradient updates within the DDPG algorithm, the raw action space of the agent is bounded within an m-dimensional vector and normalized.

$${\mathit{A}}_t=\left\{{\mathit{a}}_t\in {ℝ}^m\left|-1\le a_{j,t}\le 1,j=1,2,\dots ,m\right.\right\}$$

(21)

$$m=n_{\mathrm{MT}}+n_{\mathrm{ESS}}+n_{\mathrm{EV}}$$

(22)

where n_MT, n_ESS, and n_EV represent the number of dispatchable MT, ESS, and EVCS.

The normalized output a_t is subsequently transformed into the physical control vector u_t via a linear inverse mapping function f_map(·):

$${\mathit{u}}_t=f_{\mathrm{map}}\left({\mathit{a}}_t\right)={\left[{\mathit{P}}_t^{\mathrm{MT}},{\mathit{P}}_t^{\mathrm{ESS}},{\mathit{\alpha }}_t^{\mathrm{EV}}\right]}^{\top }$$

(23)

where ${\mathit{P}}_t^{\mathrm{MT}}$, ${\mathit{P}}_t^{\mathrm{ESS}}$, and ${\mathit{\alpha }}_t^{\mathrm{EV}}$ are vectors representing the output power of MT, the charging and discharging power of ESS, and the reduction rate of EVCS.

And the sub-vector for EVCS load curtailment ratios ${\alpha }_{j,t}^{\mathrm{EV}}$ is mapped under a flexibility margin α_max:

$${\alpha }_{j,t}^{\mathrm{EV}}={\displaystyle \frac{a_{j,t}+1}{2}}\cdot {\alpha }_{\max },j\in n_{\mathrm{EV}},\forall {\alpha }_{j,t}^{\mathrm{EV}}\in {\mathit{\alpha }}_t^{\mathrm{EV}}$$

(24)

This mathematical bounding limits the agent’s load-shedding capacity within [0, α_max], preventing the exploitation of constraint violations.

(3)

Reward function.

The reward function is designed according to the optimization objective of the ADN dispatch problem. Since the reinforcement learning agent aims to maximize the cumulative reward, the reward is formulated as the negative value of the comprehensive objective function. Therefore, a larger reward corresponds to a lower economic cost and fewer security violations.

The single-step reward R_t is formulated as the negative sum of the comprehensive economic cost and the quadratic security penalties:

$$R_t=-\left(C_{\mathrm{eco},t}+{\lambda }_{\mathrm{v}}{P}_{\mathrm{v},t}+{\lambda }_{\mathrm{c}}{P}_{\mathrm{c},t}\right)$$

(25)

Specifically, C_eco,t represents the economic operating cost, including the power exchange cost, MT fuel cost, ESS degradation cost, and EVCS flexibility compensation cost. P_v,t and P_c,t represent voltage deviation and operational constraint penalties, respectively. The penalty coefficients λ_v and λ_c regulate the trade-off between economic performance and operational security.

3.2. Topology-Aware State Extraction via Multi-Head GAT

Standard MLPs inherently flatten the input state, thereby destroying the spatial adjacency information of the power grid. To enable topology awareness, the ADN is modeled as a directed graph G = (V,E), where V denotes the set of buses and E represents the distribution lines.

Based on the decoupled state space, the global features ${\mathit{X}}_t^{\mathrm{glb}}$ explicitly bypass the graph attention layers to prevent meaningless message passing, while the nodal feature matrix ${\mathit{X}}_t^{\mathrm{node}}$ is directly fed into the spatial feature extractor.

Specifically, the nodal features and topology information are processed by the multi-head graph attention network to obtain topology-aware spatial representations:

$${\mathit{H}}_t^{\mathrm{GAT}}=\mathrm{GAT}({\mathit{X}}_t^{\mathrm{node}},\mathit{A})$$

(26)

where A denotes the adjacency matrix describing the connectivity relationship among buses, and ${\mathit{H}}_t^{\mathrm{GAT}}$ represents the extracted topology-aware node embeddings.

After extracting topology-aware node representations, the global features and spatial features are concatenated to construct the final state representation for the reinforcement learning agent:

$${\mathit{Z}}_t=[{\mathit{X}}_t^{\mathrm{node}}||{\mathit{H}}_t^{\mathrm{GAT}})$$

(27)

where Z_t denotes the topology-aware state representation that is subsequently fed into the Actor–Critic networks.

To capture the dynamically shift in spatial correlations, a multi-head GAT is utilized to extract spatial correlations from ${\mathit{X}}_t^{\mathrm{node}}$. For a specific attention head k, the attention coefficient $e_{ij}^{\left(k\right)}$ indicating the influence of node j on node i is computed as follows:

$$e_{ij}^{\left(k\right)}=\mathrm{LeakyReLU}\left({\mathit{a}}^{\left(k\right)\mathrm{T}}\left[{\mathit{W}}^{\left(k\right)}{\mathit{x}}_i‖{\mathit{W}}^{\left(k\right)}{\mathit{x}}_j\right]\right)$$

(28)

where W^(k) is a learnable shared linear transformation matrix; x_i, x_j represent the node feature vectors of node i and node j, respectively; a^(k) is the weight vector of the attention mechanism, and $\left|\right|$ denotes the concatenation operation.

Based on this, the coefficients are normalized using the softmax function over the topological neighborhood:

$${\alpha }_{ij}^{\left(k\right)}={\displaystyle \frac{\exp \left(e_{ij}^{\left(k\right)}\right)}{{\displaystyle \sum _{l\in {\mathcal{N}}_i}\exp \left(e_{il}^{\left(k\right)}\right)}}}$$

(29)

where ${\alpha }_{ij}^{\left(k\right)}$ represents the normalized attention weight of node j to node i under the kth attention head.

The outputs of K independent attention heads are concatenated and passed through an exponential linear unit (ELU) alongside a residual connection, yielding the updated node representations h′_t:

$${\mathit{h}}_i^{\prime }=\mathrm{ELU}\left({\mathit{W}}_{\mathrm{proj}}\left({\Vert }_{k=1}^K{\displaystyle \sum _{j\in {\mathcal{N}}_i}{\alpha }_{ij}^{\left(k\right)}{\mathit{W}}^{\left(k\right)}{\mathit{x}}_j}\right)+{\mathit{x}}_i\right)$$

(30)

where $|{|}_{k+1}^k$ represents the concatenation operation. After GAT layers L, the updated node representations are flattened to preserve the spatial position of each bus. The flattened spatial feature is then concatenated with the global feature embedding and fed into the Actor and Critic networks. In this way, the dispatch policy can make continuous control decisions based on both system-level operating conditions and topology-aware nodal representations.

3.3. Centralized Actor–Critic Training Paradigm

Based on the topology-aware state representation Z_t obtained from the multi-head GAT, a centralized Actor–Critic framework based on Deep Deterministic Policy Gradient (DDPG) is adopted to optimize the continuous-dispatch strategy. The Actor network generates the continuous control actions according to the current state representation, which can be expressed as follows:

$$a_t=\mu (Z_t|{\theta }^{\mu })$$

(31)

where a_t denotes the continuous-dispatch action, μ(⋅) represents the Actor network, and θ^μ represents the parameters of the Actor network.

The Critic network evaluates the generated action by estimating the state-action value function:

$$Q(Z_t,a_t|{\theta }^Q)$$

(32)

where θ^Q denotes the parameters of the Critic network. The Critic network provides the action-value evaluation to guide the update of the Actor network.

During training, the Critic is updated by minimizing the temporal-difference error between the estimated Q-value and the target Q-value:

$$L\left({\theta }^Q\right)={\displaystyle \frac{1}{N_b}}{\displaystyle \sum _{i=1}^{N_b}{\left(y_i-Q\left(Z_i,a_i\left|{\theta }^Q\right.\right)\right)}^2}$$

(33)

where θ^u represents the parameters of the Actor network; L(θ^Q) represents the loss function of the Critic network; N_b represents the number of samples in a training batch; Z_i, a_i represents the state and action in the ith sample; Q(Z_i, a_i) represents the estimated Q-value of the current Critic network for state Z_i and action a_i; y_i represents the target Q-value.

Where the target Q-value is defined as follows:

$$y_i=r_i+\gamma Q^{\prime }\left(Z_{i+1},{\mu }^{\prime }\left(Z_{i+1}\left|{\theta }^{{\mu }^{\prime }}\right.\right)\left|{\theta }^{Q^{\prime }}\right.\right)$$

(34)

where r_i represents the immediate reward obtained by the ith sample at the current time; γ represents the discount factor, typically between 0 and 1, used to measure the importance of future rewards; μ′(Z_i+1|θ^μ′) represents the action generated by the target Actor network based on the next topology-aware state representation Z_i+1; Q′ represents the target Critic.

The Actor is updated by maximizing the expected Q-value estimated by the Critic. Equivalently, the Actor loss represents the training objective of the Actor network, which can be written as follows:

$$L\left({\theta }^{\mu }\right)=-{\displaystyle \frac{1}{N_b}}{\displaystyle \sum _{i=1}^{N_b}Q\left(Z_i,\mu \left(Z_i\left|{\theta }^{\mu }\right.\right)\left|{\theta }^Q\right.\right)}$$

(35)

where μ represents the current Actor network.

The Actor network’s role is to output an action based on the current state Z_i. It aims to output an action that will elicit a higher Q-value from the critic.

To improve training stability, experience replay is adopted to break the temporal correlation of sequential samples, and target networks are softly updated as follows:

$${\theta }^{\prime }\leftarrow \tau \theta +\left(1-\tau \right){\theta }^{\prime }$$

(36)

where θ, θ′ represents the current and target network parameters; τ is the soft update coefficient, which is usually a very small positive number.

And the Actor and Critic networks are optimized using Adam optimizer with learning rates α^u, α^Q.

Through this centralized training paradigm, the agent learns a continuous-dispatch policy that jointly considers economic operation satisfying physical constraints.

Algorithm 1 demonstrates the implementation of our proposed solution for solving the ADN energy dispatch problem, which was described in Section 3. For ease of implementation, we divide the process into two parts: (a) GAT-based feature extraction, and (b) DDPG-based decision making.

Algorithm 1. Training process of the proposed GAT-DDPG framework

Input: Historical operation scenarios; replay buffer D; Actor network parameters θμ; Critic network parameters θQ; target Actor and Critic parameters θμ′ and θQ′ Step 1: Initialize network parameters         Initialize the Actor, Critic, and corresponding target networks: θμ, θQ, θμ′, θQ′, where         θμ = θμ′, θQ = θQ′, and initialize the replay buffer D = 0

Step 2: State perception and topology-aware feature extraction         At each dispatch time t, the system state is obtained as: St = [${\mathit{X}}_t^{\mathrm{glb}}$,${\mathit{X}}_t^{\mathrm{node}}$]         The node-level features are processed by the multi-head graph attention network:         ${\mathit{H}}_t^{\mathrm{GAT}}$ = GAT(${\mathit{X}}_t^{\mathrm{node}}$, A)         The topology-aware state representation is obtained by: Zt = [${\mathit{X}}_t^{\mathrm{glb}}$, ${\mathit{H}}_t^{\mathrm{GAT}}$]

Step 3: Action generation         The Actor network generates continuous control actions: at = μ′(Zi|θμ] Step 4: Environment interaction         The next state and reward are obtained: Zt+1, rt Step 5: Experience storage         The transition tuple:(Zt, at, rt, Zt+1) is stored into the replay buffer D. Step 6: Critic network update         A mini-batch of transitions is randomly sampled from D         The target value is calculated as: yi = ri + γQ′(Zi+1, μ(Zi+1))         The Critic loss is: LQ = yi − Q(Zi, ai|θQ)2         Then, the Critic parameters are updated.

Step 7: Actor network update         The Actor objective is LQ = −Q(Zi, μ(Zi,)):

Step 8: Target network update         The target networks are updated using soft update: θ′ = τθ + (1 − τ)θ′ The above procedure is repeated until the maximum training episodes Ne or convergence criterion is satisfied.

3.4. Proposed Optimization Dispatch Considering the Balancing Responsibility

3.4.1. The Balancing Responsibility Among Multiple Entities

Based on the GAT-DDPG model proposed in this paper, the dispatch results for flexible resources can be further optimized. However, in actual power system operations, the day-ahead dispatch is typically formulated based on load forecasts, new energy output forecasts, unit operating constraints, and information submitted by market participants, utilizing SCUC and SCED. Furthermore, advancements in short-term solar and wind forecasting technology have also contributed to improvements in forecast accuracy [31,32]. Nevertheless, as the actual operating time approaches, actual new energy output and load demand may deviate from the day-ahead forecasts, thereby resulting in power imbalances within the region. This paper takes the negative deviation in photovoltaic (PV) output—specifically, instances where the forecasted value is overestimated, preventing the PV system from meeting its dispatch output—as a case study. Consequently, building upon the target baseline dispatch results generated by the GAT-DDPG model, a deviation scenario is established in which the actual PV output falls below the dispatch output.

This shortfall in power generation creates a power deviation during the real-time operation phase, necessitating that flexible resources within the region assume the corresponding balancing responsibility, such as increasing output on the generation side and decreasing output on the load side. In addition, to ensure consistency in comparisons among the various balancing responsibility this paper does not consider the market benefits obtained by these responsibilities through balancing responsibility compensation separately; instead, it combines the following photovoltaic deviations and evaluates them from the perspective of the additional operating costs of the system in Formula (2).

$$P_{i,t}^{\mathrm{PV},\mathrm{real}}={\phi }_{i,t}\cdot P_{i,t}^{\mathrm{PV},\mathrm{base}}$$

(37)

$$\Delta P_{i,t}^{\mathrm{PV}}=(1-{\phi }_{i,t})\cdot P_{i,t}^{\mathrm{PV},\mathrm{base}}$$

(38)

where $P_{i,t}^{\mathrm{PV},\mathrm{real}}$, $P_{i,t}^{\mathrm{PV},\mathrm{base}}$, $\Delta P_{i,t}^{\mathrm{PV}}$, φ_i,t represent the photovoltaic output of the ith photovoltaic period t before and after considering the deviation, the output deviation, and the deviation ratio coefficient.

Based on this framework, adjustable resources, including MTs, ESSs, and EVCSs, can exploit their operational flexibility to respond promptly to system power imbalances. By incorporating the balancing responsibility allocated to each entity at each time interval, the dispatch decisions of these flexible resources can be further optimized. And $\Delta P_{i,t}^{\mathrm{PV}}$ represents the balancing responsibility that needs to be assumed by MT, ESS, or EVC.

$$A_{k,i,t}={\xi }_{k,i,t}\times \Delta P_{i,t}^{\mathrm{PV}}$$

(39)

$${\displaystyle \sum _k^3{\displaystyle \sum _i^{I_k}{\xi }_{k,i,t}}}=1$$

(40)

where A_k,i,t, ξ_k,i,t represent the uncompensated power gap of the balancing responsibility and the balancing responsibility coefficient of the subject of subject type k in time period t, where k = 1 corresponds to MT, k = 2 corresponds to ESS, and k = 3 corresponds to EVCS; I_k is the number of entities of the kth type.

In assuming the corresponding balancing responsibility, it is necessary to consider allowing flexible loads to assume these balancing responsibility.

$$P_{k,i,t}^{\mathrm{unb}}=A_{k,i,t}-R_{k,i,t}$$

(41)

$$P_t^{\mathrm{unb}}\ge 0$$

(42)

where $P_{k,i,t}^{\mathrm{unb}}$, R_k,i,t represents the uncompensated balancing responsibility power gap and the balancing responsibility power actually borne by the ith subject of subject type k in time period t.

Furthermore, not all entities can fully compensate for the reduced photovoltaic power generation without constraints.

For MT, their compensation capacity is mainly limited by maximum output and ramp-up capability, and its cost changes ΔC_MT mainly include the fuel cost corresponding to assuming the balancing responsibility.

$$R_{1,i,t}=\min (A_{1,i,t},P_{\max }^{\mathrm{MT}}-P_{1,i,t}^{\mathrm{base}})$$

(43)

$$\Delta C_{\mathrm{MT}}={\displaystyle \sum _{t=1}^T({\lambda }_{\mathrm{MT}}\cdot R_{1,i,t}\cdot \Delta t)}$$

(44)

where $P_{k,i,t}^{\mathrm{base}}$ represents the initial economic dispatch output of the ith subject of subject type k in time period t.

When an ESS compensates for a negative PV power deviation, increasing its discharge power only during the deviation period without arranging subsequent charging will reduce its end-of-day SOC and consequently weaken its subsequent regulation capability. Therefore, an end-of-day SOC recovery constraint is further introduced, requiring the ESS to restore its SOC to the initial level by the end of the dispatch horizon.

To ensure power balance during the recovery charging process, this paper treats the additional recovery charging of ESS as new load and assumes that it is prioritized for fulfillment by the main grid during low-price periods. Thus, the additional cost ΔC_ESS of ESS assuming balancing responsibility includes not only the cost of additional discharge degradation but also the cost of purchasing electricity required to restore SOC.

$$R_{2,i,t}=\min (A_{2,i,t},P_{\max }^{\mathrm{dis}}-P_{2,i,t}^{\mathrm{base}},{\displaystyle \frac{(SO{C}_{\max }-SO{C}_{\min })\cdot {\eta }_{\mathrm{dis}}{E}_{\mathrm{cap}}}{\Delta t}})$$

(45)

$$\Delta C_{\mathrm{ESS}}={\displaystyle \sum _{t=1}^T({\lambda }_{\mathrm{ESS}}\cdot R_{2,i,t}+{\lambda }_t^{\mathrm{grid}}\cdot \frac{R_{2,i,t}}{{\eta }_{\mathrm{dis}}{\eta }_{\mathrm{ch}}})\Delta t}$$

(46)

For EVCS, its compensation capability is reflected in the ability to reduce charging load. Constrained by current charging demand, the maximum reduction ratio, and user charging demand, its cost variation mainly includes the load curtailment compensation cost ΔC_EVCS corresponding to assuming balancing responsibility. Therefore, this paper limits the actual power that each entity can bear to the smaller value between the photovoltaic negative deviation power and the entity’s available adjustment capability. When the actual power borne by the entity is less than the photovoltaic negative deviation power, the difference is defined as the uncompensated deficit, used to evaluate the feasibility of the entity assuming balancing responsibility.

$$R_{3,i,t}=\min (A_{3,i,t},({\alpha }_{\max }-{\alpha }_{i,t})\cdot P_{i,t}^{\mathrm{EV}\_\max })$$

(47)

$$\Delta C_{\mathrm{EVCS}}={\displaystyle \sum _{t=1}^T{\displaystyle \sum _{i\in {\Omega }_{\mathrm{EVCS}}}{\lambda }_i^{\mathrm{EVCS}}}\cdot R_{3,i,t}\cdot \Delta t}$$

(48)

3.4.2. Balancing Responsibility Optimization

The objective function of the balancing-responsibility-based dispatch model is formulated on the basis of the benchmark economic dispatch results. In addition to the economic operating cost, the model incorporates penalties associated with voltage violations and line congestion, while further accounting for the economic costs caused by balancing-responsibility fulfillment deviations and responsibility transfers.

$$\min J=C_{\mathrm{eco}}+{\lambda }_{\mathrm{V}}{P}_{\mathrm{v}}+{\lambda }_{\mathrm{C}}{P}_{\mathrm{c}}+{\lambda }_{\mathrm{resp}}{P}_{\mathrm{resp}}+{\lambda }_{\mathrm{tran}}{P}_{\mathrm{tran}}$$

(49)

$$P_{\mathrm{resp}}={\displaystyle \sum _{t=1}^T{\displaystyle \sum _{i=1}^{I_k}{(1-\frac{R_{k,i,t}}{A_{k,i,t}})}^2}}$$

(50)

$$P_{\mathrm{tran}}={\displaystyle \sum _{t=1}^T{\displaystyle \sum _{i=1}^{I_k}\max (A_{k,i,t}-R_{k,i,t},0)}}$$

(51)

where λ_resp, and λ_tran are the weighting factors for balancing responsibility performance deviation and responsibility transfer amount penalty, respectively; P_resp and P_tran are the responsibility performance deviation and responsibility transfer amount penalty, respectively.

It should be ensured that the allocated balancing responsibility is consistent with the balancing responsibility actually undertaken.

$${\displaystyle {\displaystyle \sum _{k=1}^3}{\displaystyle \sum _{i=1}^{I_k}{R}_{k,i,t}}}={\displaystyle {\displaystyle \sum _{k=1}^3}{\displaystyle \sum _{i=1}^{I_k}{A}_{k,i,t}}}$$

(52)

4. Case Study

4.1. Simulation Setup

The proposed method is validated on a modified IEEE 33-bus ADN, as shown in Figure 2, where the numbers denote the corresponding bus numbers. The test system integrates renewable generation, MTs, ESSs, and heterogeneous EVCSs.

The dataset is divided into training and testing sets. For each month, the first 20 days are used for training, while the remaining 125 days are selected as unseen test scenarios. The scheduling horizon is 24 h with a 1 h step. Historical renewable generation and load profiles are used to construct daily operating scenarios, while TOU electricity prices are adopted to model the economic interaction with the upstream grid. The main physical parameters of ADN components are summarized in

Table 2. Grid operating parameters.

Category	Parameter	Value	Unit
MTs (Nodes 6, 25, 31)	$P_{\mathrm{MT} \max }$	600/800/800	kW
	$P_{\mathrm{MT} \min }$	80/100/100	kW
	Ru/Rd	100/125/120	kW/h
	λMT	0.45/0.37/0.40	¥/kWh
ESSs (Nodes 17, 32)	Ecap	600	kWh
	$P_{\mathrm{ch} \max }$, $P_{\mathrm{dis} \max }$	300	kW
	ηdis,  ηch	0.95	/
	SOC0	0.5	/
	SOCmax	0.8	/
	SOCmin	0.2	/
	λESS	0.40	¥/kWh
Grid	$P_{\mathrm{grid} \max }$	3000	kW
EVCSs (Nodes 14, 20, 30)	λEVCS	0.6/0.5/0.4	¥/kWh
	αmax	0.2	/
Time Period	0:00–7:00, 23:00–24:00	0.37	¥/kWh
	7:00–9:00, 12:00–16:00, 21:00–23:00	0.69	¥/kWh
	9:00–12:00, 16:00–21:00	0.98	¥/kWh
Objective Function Weights	λv,λc	2000, 5	/
	λV, λC, λresp, λtran	2000, 5, 2, 0.1	/

, and the hype-parameters of the proposed GAT-DDPG is shown in

Table 3. Hype-parameters of the proposed GAT-DDPG.

Hyperparameter	Parameter	Value
Actor learning rate	αμ	1 × 10−4
Critic learning rate	αQ	1 × 10−3
Discount factor	γ	0.99
Soft update coefficient	τ	0.005
Replay buffer size	Nb	100,000
Training episodes	Ne	1500
Batch size	B	256
GAT layers	L	2
GAT attention head	K	2

.

4.2. Convergence Analysis

To evaluate the training performance of the proposed topology-aware architecture, three DDPG-based methods are compared under the same training settings:

Method 1: MLP-DDPG, which uses a multilayer perceptron to process the flattened state vector and serves as the topology-free baseline.

Method 2: GCN-DDPG, which uses graph convolution based on the fixed feeder adjacency matrix and serves as the static topology-aware baseline.

Method 3: GAT-DDPG, which is the proposed method and uses multi-head graph attention to learn state-adaptive topology-aware features from nodal states.

Figure 3 compares the cumulative reward curves of Method 1, Method 2, and Method 3 over 1500 training episodes. A moving average is applied to show the overall convergence trend, while the raw rewards are plotted in the background.

Method 1 shows relatively large fluctuations in the early training stage, indicating that the flattened state representation has limited ability to capture the spatial coupling among buses. In contrast, Method 2 and Method 3 achieve faster reward improvement by incorporating the feeder topology into policy learning.

The subplot for episodes 1000–1500 shows that Method 3 achieves the highest and most stable cumulative reward after convergence. Method 2 generally outperforms Method 1, but a noticeable reward drop appears around episode 1080, indicating that fixed graph aggregation may be less robust under changing operating conditions. Method 1 converges to a lower reward level, suggesting that topology-free flattened features limit the quality of the learned policy. These results demonstrate that the dynamic attention mechanism of Method 3 improves convergence stability and final dispatch performance.

4.3. Economic Dispatch Analysis

Figure 4 presents the 24 h dispatch results obtained by the proposed GAT-DDPG under a daily operating scenario, which is August 30. The proposed method maintains power balance throughout the scheduling horizon. Renewable generation is preferentially utilized to reduce operational cost, while the MTs dynamically adjust their outputs according to load demand and electricity price variations. During peak-load periods, the MTs increase their generation to reduce expensive grid power purchases. In contrast, surplus renewable energy during midday periods is partially exported to the upstream grid, providing additional economic benefits. The ESSs exhibit clear time-shifting behavior by charging during low-price periods and discharging during peak-demand intervals, thereby improving overall operational flexibility. The coordinated dispatch results demonstrate that the proposed GAT-DDPG framework can effectively capture the spatiotemporal coupling characteristics of the ADN and achieve economically efficient dispatch decisions.

The economic performance is summarized in

Table 4. Economic performance comparison.

Method	Cost/¥
Method 1	2,592,692.09
Method 2	2,645,934.68
Method 3	2,401,282.25

.

Method 2 obtains the highest operating cost among the three methods, indicating that simply incorporating static topology information does not necessarily improve dispatch performance. Although GCN-DDPG introduces feeder connectivity into the decision-making process, its fixed graph aggregation mechanism relies on a predefined adjacency matrix and cannot adaptively capture the changing spatial correlations under different operating conditions. Consequently, the extracted features may contain redundant information or fail to emphasize critical nodes during dynamic dispatch scenarios.

In comparison, Method 1 achieves better performance than Method 2 by avoiding unnecessary static topology propagation, although it lacks explicit spatial awareness. Method 3 achieves the lowest operating cost on the long-term test set. By employing multi-head graph attention, the proposed GAT-DDPG dynamically learns the importance of different node interactions according to operating conditions, thereby improving the coordinated utilization of distributed energy resources and achieving more economical dispatch decisions.

4.4. Voltage Security and Topology Awareness

Maintaining nodal voltage within the security range of 0.95–1.05 p.u. is an important constraint in ADN dispatch [33]. Figure 5 shows the voltage profiles obtained by Method 3. The 3D voltage surface indicates stable voltage distributions across all buses over the scheduling horizon. In addition, Figure 6 presents the voltage trajectories of the feeder-end nodes, where the voltages remain within the security limits under different operating conditions under the daily operating scenario.

The voltage security performance further verifies the limitations of static topology-based feature extraction.

Table 5. Voltage violation performance comparison.

Method	Total Violation Times	Maximum Voltage	Minimum Voltage
Method 1	20	1.0301	0.9481
Method 2	35	1.0290	0.9471
Method 3	7	1.0308	0.9465

compares the voltage regulation performance of the three methods on the independent test set. Consistent with the economic performance results, Method 2 exhibits the largest number of voltage violation occurrences. This indicates that the fixed topology aggregation mechanism in GCN-DDPG cannot effectively adapt to the dynamically changing spatial correlations among nodes under different operating conditions. Although feeder connectivity information is introduced, the predefined adjacency matrix may result in insufficient identification of critical nodes and ineffective propagation of voltage-related features, thereby limiting voltage regulation capability.

Method 1 achieves fewer voltage violation occurrences than Method 2, suggesting that the topology-free feature representation avoids the potential negative effects caused by inappropriate static feature aggregation, although it lacks explicit spatial awareness. In contrast, Method 3 achieves the best voltage security performance with only 7 voltage violation occurrences by employing multi-head graph attention to dynamically learn the importance of node interactions. The adaptive topology-aware representation enables more accurate identification of critical voltage variations and improves the coordinated control capability of distributed resources under diverse operating conditions.

The ideal voltage security performance of Method 3 is closely related to its state-adaptive topology-aware attention mechanism. Figure 7 shows the attention weights learned by the GAT layers under different operating conditions.

Figure 7a shows the attention matrix during the PV generation peak period. Besides the dominant self-attention on the diagonal, noticeable off-diagonal attention weights appear around several controllable resource nodes, indicating that the agent actively aggregates spatial information from neighboring flexible devices.

Figure 7b further presents the attention shift between different operating periods. The clear changes in off-diagonal attention weights demonstrate that Method 3 dynamically adjusts the importance of neighboring buses according to system states, rather than relying on fixed graph aggregation. This adaptive message-passing mechanism improves the capability of capturing state-dependent spatial correlations and potential localized operational interactions in the ADN.

4.5. Optimal Dispatch Considering the Balancing Responsibility

To address the regulation characteristics of different types of entities, this paper assigns MT, ESS, and EVCS to handle the negative photovoltaic power deviation, with φ_i,t set to 0.9 for the photovoltaic power reduction scenario and the initial balancing responsibility is the same for all entities, with ξ_k,i,t set to 0.125. MT compensates for the reduced photovoltaic power generation by increasing unit output, demonstrating strong continuous capacity but increasing fuel costs. ESS participates in deviation compensation by increasing discharge power, offering advantages in fast response and low regulation costs, but its continuous capacity is limited by SOC and energy-storage capacity constraints. EVCS participates in power balancing by reducing charging load, with its adjustability influenced by current charging load, maximum reduction ratio, and user charging demand constraints. Therefore, this paper further compares and evaluates the economy, safety, and feasibility of different entities assuming balancing responsibility from the perspectives of additional operating costs, node voltage levels, and uncompensated deficits.

Table 6. Comparison of differentiated entities bearing balancing responsibility.

Type	Node	Variation in Average Node Voltage/×10−4		Additional Costs/¥	Total Unbalanced/kWh
		Node 18	Node 33
MT	6	−9.5072	−1.6100	23.15	321.84
	25	−9.7159	−1.8217	20.44	318.02
	31	−7.6726	4.0497	149.31	0.00
ESS	17, 32	−9.8004	−1.9066	0.00	373.28
EVCS	14	−3.0038	0.0264	201.00	38.28
	20	−9.7661	−1.8720	69.89	233.49
	30	−9.6400	−1.5345	11.07	345.61

below compares the balancing responsibility undertaken by different entities. MT is limited by its rated capacity and minimum technical output, ESS is limited by its SOC constraint, and EVCS is limited by further load reduction restrictions, thus limiting their ability and potential to assume the balancing responsibility. MT31, due to its lower fuel cost of only 0.4, has a lower cost to assume the balancing responsibility than the 14-node EVCS, which still has a slight deviation in balancing responsibility. Furthermore, ESS at nodes 17 and 32 have no capacity to assume the balancing responsibility at all, as their economically dispatchable electricity is consumed before photovoltaic output is available, as shown in Figure 8.

Furthermore, the voltage only increases when MT31 and EVCS14 bear a greater responsibility for balancing. MT31 is the main generator and is close to the end node. When photovoltaic power decreases, if MT31 increases its power generation to cover the shortfall, it is equivalent to increasing active power injection near the end of the distribution network, thereby reducing the power transmitted from the main grid to the end and reducing the line voltage drop. Therefore, the voltage at Node33 will increase, and the support for the end voltage will be more significant. EVCS14 is the main load generator. Bearing the responsibility for balancing essentially means reducing the charging load. After the load decreases, the line current decreases and the voltage drop decreases, which may also cause the voltage to increase slightly.

Meanwhile, since the economic dispatch already takes into account the limits of and fluctuations in its MT, ESS, and EVCE boundaries, and floating, further considering having it bear the system imbalance caused by the reduction in photovoltaic power would mean that if it is already at the constraint boundary, it would be unable to further assume the corresponding balancing responsibility.

Therefore, combining the initially determined the balancing responsibility of each entity, and considering the economic efficiency, voltage overruns, performance deviations, and regulated power consumption in the optimized dispatch model, the balancing responsibility of different types of entities is further optimized by constraint solving, as shown in Figure 9. The red dashed line in Figure 9 represents the time-varying photovoltaic power deviation in each scheduling period. During execution, the balancing responsibility is mainly fulfilled by MTs that still have scheduling space and lower costs. Then, using the differential evolution algorithm, the economic dispatch result considering the balancing responsibility optimization is obtained, as shown in Figure 10.

5. Conclusions

This paper proposes a state-adaptive topology-aware continuous-dispatch framework via Multi-Head GAT-DDPG for ADNs. By embedding a multi-head GAT within a centralized Actor–Critic training paradigm, the proposed model adaptively updates spatial message-passing weights based on operational states, alleviating the limitation of fixed aggregation weights in traditional GCN-based approaches. Extensive validations on a 125-day unseen test set demonstrate that the framework successfully reduces comprehensive operating costs and voltage violations compared with representative baseline methods. Furthermore, visualizing the state-dependent attention shifts provides robust physical interpretability by tracking dynamically shifting network vulnerabilities, offering a reliable data-driven solution for optimal ADN dispatch. Based on this, and combined with the optimized dispatch method of balanced responsibility, the ability of different flexible resources to support safe and stable operation and the ideal dispatch results under the temporary reduction in new energy output are further simulated. Building on the benchmark economic dispatch results, a balancing-responsibility-based redispatch model is further developed to address renewable generation shortfalls. The model jointly considers economic operating costs, voltage deviations, line congestion, balancing-responsibility fulfillment deviations, and responsibility transfers while incorporating the operating constraints of MTs, ESSs, and EVCSs. The results show that different flexible resources exhibit distinct balancing capabilities because of their heterogeneous operating characteristics, available regulation margins, and electrical locations. Overall, the proposed framework provides an integrated data-driven and optimization-based approach for coordinating the economic dispatch and balancing-responsibility allocation of multiple flexible resources in ADNs under renewable generation uncertainty.

Although the proposed framework has been validated on a modified IEEE 33-bus ADN under a representative 10% PV reduction scenario, further investigations are required to evaluate its scalability and robustness under larger-scale distribution networks, diverse renewable generation deviation levels, and topology variations. Moreover, the impacts of heterogeneous flexible resources, including MTs, ESSs, and EVCSs, on balancing-responsibility allocation and coordinated redispatch should be further investigated under different resource penetration levels and operational scenarios. Future work will also consider hardware-in-the-loop experiments and field demonstrations to further verify the practical applicability of the proposed framework.