Signal-Guided Cooperative Optimization Method for Active Distribution Networks Oriented to Microgrid Clusters

Abstract

To achieve low-carbon collaborative operation of active distribution networks (ADNs) and microgrid clusters, this paper proposes a signal-guided collaborative optimization method. Firstly, a spatiotemporal carbon intensity equilibrium model (STCIEM) is constructed, overcoming the limitations of centralized carbon emission flow models in terms of data privacy and equitable distribution, and enabling distributed and precise carbon emission measurement. Secondly, a dual-market mechanism for carbon and electricity is designed to support peer-to-peer (P2P) carbon quota trading between microgrids and ADN-backed clearing, enhancing market liquidity and flexibility. In terms of scheduling strategy optimization, the multi-agent deep deterministic policy gradient (MADDPG) algorithm is incorporated into the carbon-electricity cooperative game framework, enabling differentiated energy scheduling under constraints. Simulation results demonstrate that the proposed method can effectively coordinate the operation of energy storage, gas turbines, and demand response, reduce system carbon intensity, improve market fairness, and enhance overall economic performance and robustness. The study shows that this framework provides theoretical support and practical reference for future distributed energy consumption and carbon neutrality paths.

1. Introduction

As the backbone of the energy industry, the power sector’s transition directly shapes economy-wide carbon emissions. As the core of energy conversion and transmission, the power system’s decarbonization and digital-intelligent transformation have become a key engine propelling the energy transition [1,2]. Within this architecture, coordination between the supply and demand sides is pivotal to efficient energy use and low-carbon operation [3]. In principle, power generation on the supply side is the dominant source of the system’s direct carbon emissions; reducing emissions at this stage is therefore often viewed as the most direct and effective mitigation pathway. In practice, however, demand-side electricity consumption not only directly influences generation output but also materially affects the system’s overall operational efficiency, thereby determining the scale and intensity of supply-side emissions. Consequently, a supply-side-only approach cannot deliver system-wide decarbonization. Deploying intelligent demand-side management (IDSM) to optimize the demand side can further curb total emissions, enable supply–demand coordination, and steer the power system toward comprehensive decarbonization. Accordingly, decarbonizing the power system requires not only optimization on the generation side but also carbon-emission accountability on the user side [4].

Given the fundamental differences between the supply and demand sides—in operating entities, governance methods, and business models—designing a carbon-emissions allocation mechanism that ensures fair burden-sharing is a core challenge for power-system decarbonization. Owing to its strong consistency with power-flow physics, the carbon emission flow (CEF) model [5] has become a key instrument for estimating nodal carbon intensity [6]. In active distribution networks (ADNs), CEF has been widely used for resource scheduling [7,8,9]; integrated with distributed-energy dispatch and demand-side management (DSM), it materially enhances low-carbon operational efficiency. Data-driven variants of CEF [10] further incorporate demand response (DR), enabling dynamic alignment between user behavior and prevailing carbon-intensity levels. In practical applications, distribution network operators generally have limited observability beyond the point of common coupling (PCC) [11], which defines the interface between an ADN and MGs. This limitation restricts their ability to access detailed user information within MGs. This constrains centralized CEF implementations at the distribution level and raises the risk of user-privacy leakage.

Under dynamic supply–demand conditions, coordinated operation between ADNs and multiple microgrids (MMGs) is pivotal to maximizing the utilization efficiency of distributed energy resources (DERs) [12,13,14]. Beyond their interactions with external markets, ADNs also enable intra-network energy transactions among MGs and influence user demand through adaptive pricing schemes [15]. To improve both flexibility and operational reliability, a variety of localized trading frameworks tailored for ADN–MMG ecosystems have been developed [16,17,18]. Although these methods enhance overall economic efficiency, they often neglect the complex interdependencies among participating entities, resulting in suboptimal and inequitable allocation of resources [15]. Centralized coordination strategies, moreover, are inadequate for accommodating the diverse objectives and autonomous behavior of MGs and ADNs, thereby failing to reflect the system’s inherently decentralized and self-regulated nature [19]. In addition, numerous studies treat interconnected MGs as homogeneous units, oversimplifying their heterogeneity and network constraints by applying relaxed formulations. Such abstraction disregards the diversity in MG structures, DER compositions, consumer characteristics, and the physical as well as operational limitations of the grid [20,21] Consequently, such models can yield infeasible or suboptimal solutions in practice, undermining system reliability and operational efficiency.

In power-system optimization, data-driven algorithms—leveraging massive streams of real-time and historical data—can quickly capture system dynamics and uncertainty to build fine-grained models that deliver efficient distributed resource scheduling, cost reduction, and risk management [22]. However, conventional reward-centric multi-agent deep reinforcement learning (MADRL) often neglects strict operational constraints; most data-driven methods lack explicit mechanisms to guarantee behavioral safety [23], limiting their effectiveness to unconstrained or lightly constrained settings [24]. The literature has explored several strategies to address these issues. A straightforward tactic is to penalize constraint violations directly within the reward function [25,26]. This, however, cannot strictly ensure that solutions remain inside the feasible set, nor can it preclude unsafe operations during offline training and online deployment.

To enhance safety assurance and regulatory compliance, various optimization techniques have been incorporated into reinforcement learning (RL). Typical strategies include pre-classifying control actions through supervised learning and introducing physical safety constraints to mitigate the likelihood of violations [27]. Interior-point optimization has also been employed to guide policy updates by imposing significant penalties on any constraint-breaching behavior [28]. Alternatively, some studies modify the policy-update rule itself, allowing updates only when constraints are satisfied and utilizing Trust Region Policy Optimization (TRPO) to address infeasible cases [29]. Although these techniques effectively reduce the frequency of violations, they simultaneously increase model complexity and computational burden, leaving residual safety challenges unresolved.

Overall, notable gaps persist in the literature regarding carbon-emissions allocation, the design of fair and efficient carbon–electricity markets, and constraint modeling for multi-agent reinforcement learning. First, although centralized CEF models align well with power-flow physics, their use in ADNs and clusters of MGs is impeded by data-sharing and privacy-protection barriers, hindering accurate metering in decentralized settings. Second, existing carbon–electricity market mechanisms rely excessively on centralized coordination and lack flexible peer-to-peer (P2P) trading and clearing/settlement mechanisms, limiting effective benefit-sharing among diverse actors. Third, traditional MADRL approaches for power-system optimization often overlook operational constraints and safety, yielding solutions with inadequate feasibility and robustness in real-world systems.

To address these challenges, we propose a low-carbon co-optimization framework for ADNs and clusters of MMGs. We develop a spatiotemporal carbon-intensity equalization model (STCIEM) that enables privacy-preserving, accurate carbon metering and fair allocation. At the market layer, we design an optimization mechanism that supports P2P trading across coupled carbon–electricity dual markets, together with ADN backstop clearing, to enhance market flexibility and fairness. For scheduling-policy optimization, we employ multi-agent deep deterministic policy gradients (MADDPG) with explicit constraint modeling to achieve supply–demand co-optimization in a decentralized game-theoretic setting, thereby improving system stability and decarbonization performance. The overall framework establishes a verifiable improvement path across the method–mechanism–control loop, with the comparison to existing studies shown in Table 1.

Table 1. Comparative Analysis of Prior Work and This Study.

Refs	User Data Privacy	Budget Fairness	Price Bounds	Clearing Stability	Safety Violation Rate
[1,6,11,16,21,26]	✓	×	×	×	×
[2,7,12,17,22,27]	✓	×	×	×	✓
[3,8,13,18,23,28]	×	×	×	✓	✓
[4,9,14,19,24]	✓	×	×	✓	×
[5,10,15,20,25,29,30]	✓	×	×	×	✓
This work	✓	✓	✓	✓	✓

This study makes three contributions.

• A distributed carbon-emission-flow framework grounded in STCIEM that circumvents the privacy limitations of centralized models and enables privacy-preserving, spatiotemporal allocation of emissions between ADNs and MGs.
• A carbon–electricity dual-market mechanism that supports peer-to-peer trading of carbon allowances together with ADN backstop clearing, thereby enhancing market flexibility while safeguarding allocative fairness.
• A constraint-aware multi-agent scheduling policy based on MADDPG that achieves supply–demand co-optimization under decentralized strategic interaction and improves system stability and decarbonization outcomes.

The rest of this paper is organized as follows. Section 2 details the modeling of ADNs and MG clusters and the proposed STCIEM. Section 3 presents the dual-market optimization mechanism, including the P2P carbon–electricity trading design and ADN backstop clearing. Section 4 describes the MADDPG-based multi-agent optimization strategy and constraint-modeling approach. Section 5 reports simulation case studies and comparative analyses. Section 6 concludes and discusses directions for future work.

2. Spatio-Temporal Equalization–Based Method for Computing Carbon-Emission Intensity

Traditional centralized CEF models estimate regional electricity carbon-emission factors by accounting for generation-side emissions and apportioning them to load areas according to power exchanges. In practice, the operating entities behind these areas are subject to commercial confidentiality and sectoral regulation, which discourages data sharing and, in turn, raises privacy concerns. To overcome this barrier, we adopt a distributed CEF computation framework grounded in the STCIEM [30]. The PCC functions as the central node for information exchange, safeguarding data integrity and supporting precise assessment of carbon-emission flows. Over each time interval, the dynamic carbon intensity for the ADN and individual MGs is derived as follows:

$$e_t^{ADN}={\displaystyle \frac{e_t^{UPG}{p}_{j,t}^{UPG}+{\sum }_{m\in M}({\mathbb{I}}_m-1)e_{m,t}^{MG}{p}_{m,t}^{trans}}{p_{j,t}^{UPG}+{\sum }_{m\in M}({\mathbb{I}}_m-1)p_{m,t}^{trans}-p_{ADN,t}^{loss}}},\forall j\in I^{ADN},m\in M$$

(1)

$$e_{m,t}^{MG}={\displaystyle \frac{e_t^{ADN}{p}_{m,t}^{trans}{\mathbb{I}}_m+e^G{\sum }_{j\in I^m}{p}_{j,t}^G}{p_{m,t}^{trans}{\mathbb{I}}_m+{\sum }_{j\in I^m}(p_{j,t}^G+p_{j,t}^{dis}+p_{i,t}^{PV})}},\forall m\in M$$

(2)

Here, $e_t^{ADN}$ and $e_{m,t}^{MG}$ represent the time-varying carbon intensities of the ADN and MG $m$, respectively. The indicator ${\mathbb{I}}_m$ is defined as 1 when net power flows from the ADN to MG $m{ (p}_{m,t}^{trans}\ge 0$), and 0 otherwise ${ (p}_{m,t}^{trans}<0)$. $p_{ADN,t}^{loss}$ denotes the internal active power loss of the ADN at time $t$; accordingly, the denominator of Equation (1) describes the net power supply that the ADN can actually deliver to the MGs and downstream loads during this time period.

Compared with centralized CEF approaches, STCIEM uses a decentralized scheme centered on the point of common coupling. It requires only periodic updates from the PCC based on time-interval power measurements. Each participant computes its carbon intensity locally with private operational parameters. The result is broadcast through the PCC. Unit-level sensitive data are not exposed, while system-level carbon-budget consistency is maintained. This design reduces reliance on global real-time synchronization. It eases communication and computation bottlenecks. It also scales well to multiple microgrids and improves robustness. The active distribution network aggregates node results using a unified weighted-average carbon intensity on a regular cadence. This mitigates the bias often seen in centralized apportionment. It strengthens consistency and fairness in settlement. Overall, STCIEM provides clear advantages in privacy protection, operational efficiency, scalability, and robustness. It also better matches the operational reality of distributed multi-agent energy systems.

3. Carbon–Electricity Dual-Market Optimization for ADN and MGs

To jointly deliver economic operation and carbon mitigation, we augment the baseline two-layer ADN–MG coordination model with a carbon-allowance trading module and cast it as a coupled carbon–electricity dual market that supports peer-to-peer bilateral negotiation together with an ADN backstop-clearing facility. The hourly electricity-trading routine is retained, and a daily carbon-allowance settlement cycle is introduced. MGs reconcile their carbon positions either via bilateral matching on the platform or by transacting directly with the ADN.

3.1. Dual-Market Optimization Model for the ADN Operator

Within the proposed architecture, the ADN operator serves as the coordinator of power dispatch and the backstop clearer in the carbon market. Its objective is to maximize the system-level net benefit while meeting the distribution network’s own decarbonization target. The corresponding optimization objective is formulated as follows:

$$max\sum _{t=1}^T\left(\mathsf{\alpha }\right({\displaystyle \frac{{\sum }_{m\in M}\left({\lambda }_t^{IM}-{\lambda }_t^{EM}\right)\left(p_{m,t}^{trans}-{\lambda }_t^{EM}{p}_{ADN,t}^{loss}\right)}{{profit}_{base}}})+\mathsf{\beta }\left(1-{\displaystyle \frac{e_t^{ADN}}{e_t^{UPG}}}\right)+(1-\mathsf{\alpha }-\mathsf{\beta })\sum _{d\in D}\sum _{m\in M}({\lambda }_{m,d}^{sell\left(ADN\right)}·Q_{m,d}^{sell\left(ADN\right)}-{\lambda }_{m,d}^{buy\left(ADN\right)}·Q_{m,d}^{buy\left(ADN\right)}))$$

(3)

An optimization model is developed for the ADN operator whose objective function contains three components: the economic surplus from electricity trading, captured by the price spread between internal and external markets; a cleanliness term that reflects the distribution network’s carbon-intensity level; and a carbon-allowance trading term that enables flexible allowance transactions among MGs and couples them with power trades. This design not only maximizes the ADN’s overall welfare but also leverages the allowance market to drive additional emissions reductions, improving abatement performance and fostering a more sustainable allocation of energy. The formal properties of the proposed STCIEM, including budget balance, fairness, uniqueness, and privacy guarantees, are provided in Supplementary S1.

$$p_{i,t}=v_{i,t}\sum _{j=1}^N{v}_{j,t}\left(G_{ij}\mathrm{c}\mathrm{o}\mathrm{s}{\delta }_{ij,t}+B_{ij}\mathrm{s}\mathrm{i}\mathrm{n}{\delta }_{ij,t}\right)$$

(4)

$$q_{i,t}=v_{i,t}\sum _{j=1}^N{v}_{j,t}\left(G_{ij}\mathrm{s}\mathrm{i}\mathrm{n}{\delta }_{ij,t}-B_{ij}\mathrm{c}\mathrm{o}\mathrm{s}{\delta }_{ij,t}\right)$$

(5)

$$p_{i,t}=p_{i,t}^{UPG}+p_{i,t}^{PV}+p_{i,t}^{GT}+p_{i,t}^{BESS}+p_{i,t}^{trans}-p_{i,t}^l-x_{i,t}^l$$

(6)

$$q_{i,t}=q_{i,t}^{UPG}+q_{i,t}^{GT}+q_{i,t}^{trans}-q_{i,t}^l$$

(7)

$$\sqrt{{\left(p_{ij,t}\right)}^2+{\left(q_{ij,t}\right)}^2}\le S_{ij}^{\mathrm{m}\mathrm{a}\mathrm{x}}$$

(8)

$$V_i^{\mathrm{m}\mathrm{i}\mathrm{n}}\le v_{i,t}\le V_i^{\mathrm{m}\mathrm{a}\mathrm{x}}$$

(9)

$$I_{ij}^{\mathrm{m}\mathrm{i}\mathrm{n}}\le i_{ij,t}\le I_{ij}^{\mathrm{m}\mathrm{a}\mathrm{x}}$$

(10)

Due to the relatively high resistance-to-reactance ratio of the (ADN–MMG system, an AC power flow model is required. Equations (4)–(7) define the network power-flow constraints, where ${\delta }_{ij,t}$ denotes the voltage angle difference between buses $i$ and $j$; $G_{ij}$ is the conductance of the line connecting buses $i$ and $j$; and $B_{ij}$ is the corresponding susceptance. When $p_{i,t}^{trans}$ ≥ 0, the power flow is directed from the distribution network to the MGs; in this case, the transferred power is treated as load from the perspective of the distribution network and as generation from the perspective of the microgrids. Equation (8) represents the line capacity constraint, Equation (9) specifies the bus voltage magnitude limits, and Equation (10) imposes the branch current magnitude constraint.

3.2. Dual-Market Optimization Model for MG Operators

Over each daily scheduling horizon, every MG performs hour-by-hour dispatch and, at day’s end, settles its carbon-allowance position against realized emissions and its initial allocation to ensure full coverage. Allowances can be traded on the platform either bilaterally with other MGs or directly with the ADN under a non-uniform-price, over-the-counter negotiation mechanism. Over each daily scheduling horizon, electricity is settled on an hourly basis, while carbon allowances are cleared once at the end of the day. In this work, daily carbon quotas are allocated and settled independently for each day, and interday carry-over of unused allowances is not considered. The optimization aims to minimize the total expenditure associated with electricity procurement and carbon trading, which can be expressed as:

$$\begin{array}{ll}min{\displaystyle \sum _{t=1}^T}({\lambda }_t^{IM}{p}_{m,t}^{trans})& +{\displaystyle \sum _{i\in I^m}}({\lambda }^G+{\lambda }^K\left|E_i^{K,t}-E_i^{K,t-1}\right|)\\ & +{\lambda }^C{e}_{m,t}^{MG}\left(p_{i,t}^{ch}+p_{i,t}^h+x_{i,t}+p_{m,t}^{loss}\right)\\ & +{\displaystyle \sum _{d\in D}}{\displaystyle \sum _{n\in M,n\ne m}}({\lambda }_{n\to m,d}^{buy}·Q_{m,d}^{buy}-{\lambda }_{m\to n,d}^{sell}·Q_{m\to n,d}^{sell})\\ & +{\displaystyle \sum _{d\in D}}({\lambda }_{m,d}^{buy\left(ADN\right)}·Q_{m,d}^{buy\left(ADN\right)}-{\lambda }_{m,d}^{sell\left(ADN\right)}·Q_{m,d}^{sell\left(ADN\right)}),\forall m,n\\ & \in M\end{array}$$

(11)

The model is subject to constraints (12)–(24). The objective in (11) comprises four components: the first two quantify, respectively, MG m operating cost over the scheduling horizon T and its carbon-emission cost; the latter two capture revenues from carbon-allowance trades among MGs and from trades between MG and the ADN. Any emissions by an MG that exceed its allocated allowances must be covered by purchasing additional allowances, whereas surplus allowances may be sold to generate income.

• First item Micro-gas-turbine operational constraints

$$Q_{m,d}=Q_{m,d}^{init}+\sum _{n\ne m}{Q}_{n\to m,d}+Q_{m,d}^{buy\left(ADN\right)}-\sum _{n\ne m}{Q}_{m\to n,d}-Q_{m,d}^{sell\left(ADN\right)}$$

(12)

$$e_{m,d}^{MG}=\sum _{t\in T}{e}_{m,t}^{MG},e_{m,d}^{MG}\le Q_{m,d},\forall m\in M,t\in T$$

(13)

$$\sum _m(\sum _{n\ne m}{Q}_{m\to n,d}+Q_{m,d}^{sell\left(ADN\right)})=\sum _m(\sum _{n\ne m}{Q}_{n\to m,d}+Q_{m,d}^{buy\left(ADN\right)}),\forall m\in M,t\in T,d=|T|$$

(14)

$$Q_{m\to n,d}\ge 0, Q_{n\to m,d}\ge 0, Q_{m,d}^{buy\left(ADN\right)}\ge 0, Q_{m,d}^{sell\left(ADN\right)}\ge 0$$

(15)

Equations (12)–(15) specify the carbon-allowance constraints. Equation (12) enforces period-by-period account balance for each MG $m$. Equation (13) limits, for every period $t$, the allowance demand of MG $m$ so that it does not exceed the total allowances available to it. Equation (14) imposes market clearing for bilateral trades, requiring that the volume of allowances sold by some MG $m$ equals the volume purchased by its counterparty $n$. Equation (15) imposes non-negativity on the allowance variables.

2.

Micro-gas-turbine operational constraints

$$P^{G,min}\le P_{i,t}^G\le P^{G,max},\forall \mathrm{i}\in I^m,\forall m\in M$$

(16)

$$∆P_{i,t}^G=P_{i,t}^G-P_{i,t-1}^G,\forall i\in I^m,\forall m\in M$$

(17)

$$\left|∆P_{i,t}^G\right|\le R^G,\forall i\in I^m,\forall m\in M$$

(18)

Equation (16) specifies the micro-gas turbine’s power-output limits, while Equations (17) and (18) enforce its ramping (up/down rate) constraints.

3.

DR operational constraints

In this study, DR is modeled under a daily energy-neutrality assumption total daily load remains constant. Let $x_{i,t}^l$ denote the load adjustment scheduled for MG $m$ at time $t$, and $p_{i,t}^l$ the baseline demand of MG $m$ at node $i$.

$$\sum _{t=1}^T\sum _{i\in I^m}{x}_{i,t}^l=0,\forall i\in I^m,m\in M$$

(19)

$$\sum _{i\in I^m}(p_{i,t}^l+x_{i,t}^l)\ge 0,\forall i\in I^m,m\in M$$

(20)

$$\left|x_{i,t}^l\right|\le {\delta }_m{p}_{i,t}^l,\forall i\in I^m, m\in M$$

(21)

Equation (19) enforces daily energy-neutrality for DR—the aggregate load over the day equals the baseline. Equation (20) requires that the post-DR total load remain nonnegative at all times. Equation (21) bounds the magnitude of load adjustments undertaken during the DR process.

4.

Energy-storage system operational constraints

$$E_{i,t+1}^K=E_{i,t}^K+{\eta }^{ch}{P}_{i,t}^{ch}+{\displaystyle \frac{1}{{\eta }^{dis}{P}_{i,t}^{dis}}},\forall i\in I^m, m\in M$$

(22)

$$0\le E_{i,t}^K\le E^{Cap},\forall i\in I^m, m\in M$$

(23)

$$P_t^{ch,min}\le \left|P_{i,t}^{ch}\right|\le P_t^{ch,max},P_t^{dis,min}\le \left|P_{i,t}^{dis}\right|\le P_t^{dis,max}, \forall i\in I^m, m\in M$$

(24)

Equation (22) describes the variation process of charging and discharging power, while Equation (23) constrains the available power range. Meanwhile, Equation (24) specifies the upper and lower limits of the charging and discharging outputs. In this paper, constraints (22)–(24), together with penalizing boundary violations in the reward function, ensure that the state of charge of the storage remains within the interval [5, 95].

3.3. P2P Carbon-Trading Mechanism

To enable more flexible and efficient carbon allowance transfers within microgrid clusters, this study develops a P2P-based carbon trading platform. MGs submit bids and offers according to their abatement capability, emission surplus or deficit, and economic preferences, and transactions are cleared via bilateral negotiation within price bounds linked to an external reference, without disclosing unit-level sensitive data. This design reflects heterogeneous marginal abatement costs, constrains arbitrage, and improves price interpretability and compliance. A unified clearing framework aligns hourly electricity settlement with daily carbon settlement, while the active distribution network serves as a residual clearer under low-liquidity conditions to maintain continuous matching and price stability. Overall, compared with centralized schemes, the platform enhances privacy, supports marginal-cost-based allocation, and improves clearing robustness and social welfare.

In each time step $t$, the ADN broadcasts the internal retail price adjustment (${\lambda }_{IM}\left(t\right)$) and the external price signal (${\lambda }_{EM}\left(t\right)$). Each MG then optimizes its local dispatch and submits a bid vector with its net demand/supply and carbon buy/sell quantities. The electricity market is cleared hourly by balancing the aggregate net demand and supply of all MGs; residual power is exchanged with the upstream grid at (${\lambda }_{EM}\left(t\right)$), and MG-side settlements use (${\lambda }_{IM}\left(t\right)$). Carbon trades are accumulated over the day and cleared once at day-end, consistent with daily carbon-quota settlement without interday carry-over. Communication in each step is limited to broadcasting a few scalar prices and collecting one bid vector per MG, so the overhead grows roughly linearly with the number of MGs and remains modest. Simulations assume perfect information synchronization and instantaneous market clearing, i.e., communication and computation delays are ignored.

3.3.1. Bidding and Clearing Rules

For each daily carbon-trading round $d\in \mathrm{D}$, every participating MG submits:

• A sell quote (ask): the amount of allowances offered $Q_{m\to n,d}\ge 0$ and the minimum acceptable unit price ${\lambda }_{m,d}^{ask}$;
• A buy quote (bid): the amount of allowances requested $Q_{n\to m,d}\ge 0$ and the maximum willingness-to-pay ${\lambda }_{m,d}^{bid}$.

The platform clears a bilateral trade between $m$ and $n$ if the admissibility condition is satisfied.

$${\lambda }_{m,d}^{bid}\ge {\lambda }_{m,d}^{ask}$$

(25)

When a trade is executed, the transaction price is set to the arithmetic mean of the two quotes:

$${\lambda }_{m\to n,d}^{{CO}_2}={\displaystyle \frac{{\lambda }_{m,d}^{ask}+{\lambda }_{m,d}^{bid}}{2}}$$

(26)

3.3.2. ADN Participation and Clearing Functionality

To enhance market liquidity and trading feasibility, the ADN is introduced as the clearing entity for carbon-allowance transactions. When trades among MGs cannot be fully matched, the platform allows bilateral transactions between MGs and the ADN to ensure a balanced supply and demand of carbon allowances within the market.

Similar to MG-to-MG transactions, MG–ADN trades also follow a bilateral quotation scheme. MG m submits to the ADN either a bid price ${\lambda }_{m,d}^{bid\left(ADN\right)}$ or an ask price ${\lambda }_{m,d}^{ask\left(ADN\right)}$. Once the trading condition is satisfied, the clearing price is determined according to the arithmetic mean rule:

$${\lambda }_{m,d}^{buy\left(ADN\right)}={\displaystyle \frac{{\lambda }_{m,d}^{ask\left(ADN\right)}+{\lambda }_{ADN,d}^{bid}}{2}}$$

(27)

$${\lambda }_{m,d}^{sell\left(ADN\right)}={\displaystyle \frac{{\lambda }_{m,d}^{bid\left(ADN\right)}+{\lambda }_{ADN,d}^{ask}}{2}}$$

(28)

The ADN’s quotation can either be fixed as a platform-guided reference or self-adjusted according to its operational strategy, thereby guiding market behavior and improving system-wide decarbonization efficiency.

The proposed P2P negotiated pricing with ADN backstop clearing has sound market properties in terms of individual rationality, budget balance, price bounds and liquidity. A trade is executed only when the buyer’s bid is no lower than the seller’s ask, and the clearing price is their arithmetic mean. Thus, the price always lies within the submitted interval, the buyer never pays more than its bid, and the seller never receives less than its ask, ensuring individual rationality. For every transaction the buyer’s payment equals the seller’s revenue; when the ADN acts as counterparty it follows the same rule, so the mechanism is budget-balanced.

All quotes are restricted to a band linked to the external carbon-market reference price, and clearing prices are clipped to this band, creating explicit upper and lower bounds and reducing scope for risk-free arbitrage. On the liquidity side, when microgrids behave as price takers, the ADN’s standing orders within this band provide a permanent liquidity channel that absorbs residual demand or supply and prevents clearing interruptions. When some microgrids have partial pricing power, the average-price rule and price bounds limit price manipulation, while the ADN backstop supplies basic depth under thin-liquidity conditions and stabilizes welfare outcomes.

4. Multi-Agent Learning Framework and Scheduling-Strategy Optimization

In the collaborative environment of carbon–electricity dual markets, each MG within the system can be regarded as an autonomous agent with coupled interactions. Its scheduling behavior is influenced not only by its own load, generation state, and carbon-emission capacity, but also by the dynamic feedback of other participants’ carbon-trading strategies. To achieve system-level coordination under limited communication and local-information constraints, a partially observable Markov game (PoMG) framework is established to represent the interactive decision process among multiple MGs. The MADDPG algorithm is employed to derive the optimal joint policy for this game. Building upon conventional deterministic policy-gradient methods, MADDPG integrates an attention mechanism to model inter-agent policy dependencies and adopts a dual-policy update structure to enhance learning stability. During training, each agent independently optimizes its policy network, while centralized training and distributed execution enable effective coordination of multi-agent strategies.

4.1. Multi-Agent Markov Game Formulation

Consider a multi-agent system composed of $N$ interconnected MGs operating within the coupled carbon–electricity markets. The interactive dynamics are formulated as a PoMG, represented by a five-tuple:

$$\mathcal{G}=⟨\mathcal{N},\mathcal{S},\left\{{\mathcal{A}}^n\right\},\left\{{\mathcal{O}}^n\right\},\left\{r^n\right\}⟩$$

(29)

where $\mathcal{N}=\left\{\mathrm{1,2},\dots ,N\right\}$ denotes the set of agents (including each MG and the ADN), $\mathcal{S}$ is the global state space that incorporates system load, power generation, carbon-allowance allocation, emission levels, and trading history; ${\mathcal{A}}^m$ represents the action space of agent m, encompassing both power-dispatch and carbon-trading decisions; ${\mathcal{O}}^n$ denotes the observation space of agent $n$; and $r^n$ is the reward function, capturing the single-period economic profit and decarbonization benefit of agent $n$.

At each time step $t$, the system state $s_t\in S$ evolves dynamically under the joint actions of all agents, denoted by $a_t=(a_t^1,a_t^2,\dots ,a_t^N)$, following the Markov transition process:

$$s_{t+1}~P(s_{t+1}\left|s_t,a_t)\right.$$

(30)

Because each agent can only access its own local observation $o_t^n=O^n\left(s_t\right)$, the corresponding policy function is defined as:

$${\pi }^n:{\mathcal{O}}^n\to {\mathcal{A}}^n$$

(31)

That is, each agent $n$ determines its action $o_t^n$ based solely according to its locally observed information $a_t$.

The goal of each agent is to maximize its long-term expected return, evaluated as the discounted cumulative reward.

$${max}_{{\pi }^n}\mathbb{E}\left[\sum _{t=1}^T{\gamma }^t{r}_t^n\right]$$

(32)

where $\gamma \in \left(\mathrm{0,1}\right)$ denotes the discount factor that determines the relative importance of future rewards.

4.1.1. Observation

At each time step, the observations of the ADN and each heterogeneous MG agent are represented as a high-dimensional vector, defined as follows:

$$o_t^n=\left\{\begin{array}{c}\left({\displaystyle \sum _{m\in M}}{P}_{m,t}^{trans},P_t^{UPG},{\lambda }_t^{EM},e_t^{ADN},e_t^{UPG},{\left\{{\lambda }_{m,d}^{ask},{\lambda }_{m,d}^{bid},Q_{m,d}\right\}}_{m\in M}\right)\in o_t^n,\forall n\in N^{ADN}\\ \left(P_{m,t}^{trans},{\displaystyle \sum _{i\in I^m}}{P}_{i,t}^l,{SOC}_{i,t}^K,{\lambda }_t^{IM},e_t^{ADN},e_{m,t}^{MG},{\lambda }_{m,d}^{ask},{\lambda }_{m,d}^{bid},Q_{m,d}\right)\in o_t^n,\forall i\in I^m,\mathrm{n}\in N^m,m\in M\end{array}\right.$$

(33)

In this context, $o_t^n$ is defined as the observation vector containing information available only at the PCC level, including internal variables that describe each entity’s operating conditions and PCC-level data that convey power-exchange information at the interface.

For the ADN, the observation includes the upstream-grid (UPG) trading power $P_t^{UPG}$, carbon intensity $e_t^{UPG}$, external electricity price ${\lambda }_t^{EM}$, total exchanged power within the system ${\sum }_{m\in M}{P}_{m,t}^{trans}$, and the ADN’s own emission intensity $e_t^{ADN}$. On this basis, carbon-trading-related variables are further incorporated, including each MG’s ask price ${\lambda }_{m,d}^{ask}$, bid price ${\lambda }_{m,d}^{bid}$, and allowance quantity $Q_{m,d}$, which serve as inputs for market matching and ADN clearing.

For each MG, the observation vector includes its purchased power $P_{m,t}^{trans}$, load power ${\sum }_{i\in I^m}{P}_{i,t}^l$, state of charge ${SOC}_{i,t}^K$, external electricity price ${\lambda }_t^{IM}$, ADN carbon intensity $e_t^{ADN}$, and local emission intensity $e_{m,t}^{MG}$. In terms of carbon trading, each MG additionally submits its own bid/ask information ${\lambda }_{m,d}^{ask},{\lambda }_{m,d}^{bid}$, and the allowance quantity $Q_{m,d}$ for the current trading period.

4.1.2. Action

For each MG $m\in M$, the action vector at time t, denoted as $a_t^n$, consists of the following variables:

$$a_t^n=(P_{n,t}^{ch},P_{n,t}^{dis},P_{n,t}^{GT},x_{n,t}^{DR},{\lambda }_{m,d}^{ask},{\lambda }_{m,d}^{bid},Q_{m,d}^{sell\left(ADN\right)},Q_{m,d}^{buy\left(ADN\right)},{\left\{Q_{m\to n,d},Q_{n\to m,d}\right\}}_{n\ne m})$$

where $P_{n,t}^{GT},x_{n,t}^{DR},P_{n,t}^{BESS}\in \left[-\mathrm{1,1}\right]$. $P_{n,t}^{ch},P_{n,t}^{dis}$ represent the charging and discharging power of the energy storage system, respectively. $P_{n,t}^{GT}$ denotes the micro gas turbine output, while $x_{n,t}^{DR}$ indicates the demand response participation level. $Q_{m,d}^{sell\left(ADN\right)}, Q_{m,d}^{buy\left(ADN\right)}$ represent the carbon allowances sold to and purchased from the ADN by MG m. The set ${\left\{Q_{m\to n,d},Q_{n\to m,d}\right\}}_{n\ne m}$ represents bilateral carbon allowance transactions among MGs, with the trading period indexed by $d$.

As the coordinator of the carbon–electricity market, the ADN’s strategy variables emphasize platform-level guidance and clearing control.

Its action vector $a_t^n$ is defined as:

$$a_t^n=({\lambda }_t^{IM},{\left\{{\lambda }_{m,d}^{buy\left(ADN\right)}\right\}}_m,{\left\{{\lambda }_{m,d}^{sell\left(ADN\right)}\right\}}_m,optional:Q_{m,d}^{init})$$

(34)

where ${\lambda }_t^{IM}$ is the internal retail price; ${\lambda }_{m,d}^{buy\left(ADN\right)}$ denotes the ADN’s bid price for carbon allowances purchased from MG ${\lambda }_{m,d}^{sell\left(ADN\right)}$ represents the ADN’s ask price for carbon allowances sold to MG.

4.1.3. State Transition

The state-transition process is affected by both control actions and stochastic disturbances in multiple state variables, including solar irradiance influencing photovoltaic generation, fluctuations in market prices, variations in UPG carbon intensity, and dynamic changes in MG loads. The transition function $P(s_{t+1}\left|s_t,a_t)\right.$ is formulated in a data-driven way to reflect the intertwined dynamics of carbon-allowance trading, power scheduling, and energy-storage management.

4.1.4. Reward Function

At each time step $t$, the reward function associated with every agent in the ADN–MMGs framework embodies the objectives of both the ADN and MG models, as formulated in Equations (3) and (11). For an ADN agent, the reward accounts for the power cost or profit $r_{n,t}^{trans}$, the benefit obtained from electricity trading, and the carbon-market clearing gain $r_{n,t}^{Carbon}$.

4.2. Multi-Agent Optimization Based on MADDPG

In the ADN–MMG cooperative optimization problem studied here, each microgrid is modeled as an independent agent operating in a continuous action space for both dispatch and trading, which makes traditional discrete-action Q-learning inadequate. We therefore adopt a MADDPG algorithm with a centralized training and decentralized execution (CTDE) scheme: global information is used during training to improve sample efficiency and convergence, while execution relies only on local observations, reducing communication overhead and protecting private operating data. Continuous policies avoid the precision loss and curse of dimensionality associated with action discretization, and feasible-set projection plus penalty terms at the policy output enforce storage SOC limits, demand response energy neutrality and bounds, and gas turbine output and ramping constraints, while naturally coupling with electricity price and carbon intensity signals. The framework has low dependence on an exact system model, can accommodate device heterogeneity, incomplete information, and uncertainty, and thus offers good scalability and practical applicability.

In a Markov game involving $N$ intelligent agents, at each time step $t$, the environment is in a global state $s_t$. The $i$-th agent observes only local information $o_{i,t}=O_i\left(s_t\right)$, and generates actions deterministically according to a strategy $a_{i,t}={\mu }_{{\theta }_i}\left(o_{i,t}\right)$. The joint action $a_t=(a_{1,t},\dots ,a_{N,t})$ influences the environment, which in turn results in a state transition probability $p\left(s_{t+1}\right|s_t,a_t)$ and an individual reward $r_{i,t}=r_i(s_t,a_t)$.

MADDPG employs Centralized Training, CTDE. During execution, each agent relies solely on its own observation $o_i$ and the actor ${\mu }_{{\theta }_i}$ associated with it. During training, however, each agent is assigned a critic that has access to joint information, which is used to evaluate the value of joint actions and stabilize multi-agent nonstationarity.

Specifically, the critic of the $i$-th agent is denoted as $Q_{{\varphi }_i}(x,a$), where $x$ is the concatenation of the local observations $x=concat(o_1,\dots ,o_N)$ and $a=(a_1,\dots ,a_N)$ is the joint action. During training, the critic uses samples from the replay buffer $\mathcal{D}$ with samples $(x_t,\left\{o_{j,t}\right\},\left\{a_{j,t}\right\},\left\{r_{j,t}\right\},x_{t+1},\left\{o_{j,t+1}\right\},d_t)$, and constructs the target for the Bellman equation as:

$$y_{i,t}=r_{i,t}+\gamma (1-d_t){Q_{{\varphi }_i}}^{\prime }(x_{t+1},{\mu }_{{\theta }_1}^{\prime }\left(o_{1,t+1}\right),\dots ,{\mu }_{{\theta }_N}^{\prime }\left(o_{N,t+1}\right))$$

(35)

The critic minimizes the Bellman error:

$${\mathcal{L}}_i\left({\varphi }_i\right)={\mathbb{E}}_{\mathcal{D}}\left[{\left(Q_{{\varphi }_i}\left(x_t,a_t\right)-y_{i,t}\right)}^2\right]$$

(36)

The strategy adopts a deterministic policy gradient as follows:

$${\nabla }_{{\theta }_i}\mathcal{T}\left({\theta }_i\right)={{\mathbb{E}}_{\mathcal{D}}\left[{\nabla }_{{\theta }_i}{\mu }_{{\theta }_i}\left(o_{i,t}\right){\nabla }_{a_i}{Q}_{{\varphi }_i}\left(x_t,a_1,\dots ,a_N\right)\right]}_{a_j={\mu }_{{\theta }_j}\left(o_{j,t}\right)}$$

(37)

That is, the joint action value $Q_{{\varphi }_i}$ is evaluated with respect to the action $a_i$ and propagated through ${\mu }_{{\theta }_i}$ for feedback. To enhance stability, MADDPG adopts a new target network:

$${\theta }_i^{\prime }\leftarrow \tau {\theta }_i+(1-\tau ){\theta }_i^{\prime }$$

(38)

$${\varphi }_i^{\prime }\leftarrow \tau {\varphi }_i+(1-\tau ){\theta }_i^{\prime }$$

(39)

Furthermore, during the exploration phase, the actor explores using noise and gradually refines the actions. The returned actions are trained in the continuous action space of the environment. Since the training is off-policy and can be reused, there is no need for the critic to focus on specific agents, allowing each agent to execute its own ${\mu }_{{\theta }_i}$ independently, which naturally satisfies the decentralization requirement.

To ensure that the learned policies remain compatible with the AC network constraints introduced in Section 3, the physical limits are treated as hard constraints during both training and execution. Let $a_t=\{a_{i,t}{\}}_{i\in \mathcal{N}}$ denote the joint action at time $t$, collecting all continuous control variables such as BESS charging/discharging power, micro-gas-turbine output, demand-response adjustments, and tie-line exchanges. Before being applied to the environment, $a_t$ is projected onto the device-level feasible set $\mathsf{\Omega }$ by a component-wise clipping operator

$${\tilde{a}}_{k,t}\leftarrow \mathrm{m}\mathrm{i}\mathrm{n}\left\{\mathrm{m}\mathrm{a}\mathrm{x}\right(a_k^{\mathrm{m}\mathrm{i}\mathrm{n}},a_{k,t}),a_k^{\mathrm{m}\mathrm{a}\mathrm{x}}\},\forall a_{k,t}\in a_t$$

(40)

where $a_k^{\mathrm{m}\mathrm{i}\mathrm{n}}$ and $a_k^{\mathrm{m}\mathrm{a}\mathrm{x}}$ denote the lower and upper bounds of each controllable variable. The clipped joint action ${\tilde{a}}_t$ is then used to update the system and to run an AC power-flow calculation, which returns the nodal voltage magnitudes $\mid V_{i,t}\mid$ and branch currents $\mid I_{ij,t}\mid$. Based on these results, we define violation indicators for the voltage-magnitude and thermal-limit constraints as

$$C_v(t)=\sum _i\left[\mathrm{m}\mathrm{a}\mathrm{x}\right(0,\mid V_{i,t}\mid -V_i^{\mathrm{m}\mathrm{a}\mathrm{x}})+\mathrm{m}\mathrm{a}\mathrm{x}(0,V_i^{\mathrm{m}\mathrm{i}\mathrm{n}}-\mid V_{i,t}\mid \left)\right]$$

(41)

$$C_{\mathcal{l}}(t)=\sum _{(i,j)}\mathrm{m}\mathrm{a}\mathrm{x}(0,\mid I_{ij,t}\mid -I_{ij}^{\mathrm{m}\mathrm{a}\mathrm{x}})$$

(42)

where $V_i^{\mathrm{m}\mathrm{i}\mathrm{n}}$, $V_i^{\mathrm{m}\mathrm{a}\mathrm{x}}$ and $I_{ij}^{\mathrm{m}\mathrm{a}\mathrm{x}}$ are the voltage and thermal limits specified for the ADN. For each agent, the instantaneous reward used in (41) is obtained by augmenting the economic and carbon terms (denoted $r_{i,t}^0$) with safety penalties,

$$r_{i,t}=r_{i,t}^0-{\rho }_v{C}_v(t)-{\rho }_{\mathcal{l}}{C}_{\mathcal{l}}(t)$$

(43)

where ${\rho }_v,{\rho }_{\mathcal{l}}>0$ are penalty coefficients. When all voltage and line-loading constraints are satisfied, $C_v\left(t\right)=C_{\mathcal{l}}\left(t\right)=0$ and the reward is unaffected; any violation leads to a reduction in $r_{i,t}$ proportional to the severity of the constraint breach. In this way, the projection in (40) and the penalty terms in (41)–(43) jointly discourage infeasible actions during training, while the same projection operator is applied in the execution phase to guarantee that the dispatched schedules satisfy the AC network limits.

In our implementation, the feasible set $\mathsf{\Omega }$ explicitly encodes all device-level constraints in Equations (16)–(24), including GT generation limits and ramping, DR power bounds and energy-neutrality over the scheduling horizon, and BESS power/SOC bounds. During both training and testing, all joint actions are first projected onto $\mathsf{\Omega }$, so that these constraints are enforced at every step.

Compared to the single-agent DDPG, the key difference in MADDPG lies in the input of the critic: the state feature $x$ and the joint action $a$ are included. This allows the critic to compute the value function that reflects the change in the strategies of other agents, which helps to stabilize the system. On the other hand, the actor only relies on the agent’s own observation $o_i$, ensuring that actions are taken without the need for information or communication from other agents. This structure facilitates collaboration, competition, and mixed scenarios, and it can dynamically select whether or not to share parameters. Overall, MADDPG maintains the continuous action model of DDPG while using CTDE to centralize the collection of information from multiple agents, forming a clearer and more stable multi-agent reinforcement learning framework.

5. Case Study

The study uses the IEEE 33-bus system for optimization model validation to address the low-carbon operation problem of the ADN-MMG. The configuration of DER is shown in Figure 1. In order to improve the robustness of the model under future random environmental conditions, the training process includes high-frequency noise, applying load power, the power generation capacity of photovoltaic systems, price, and the volatility of the distribution network. The training data is based on historical information, and the configuration includes AMD EPYC 9554-CPU, 128GB of memory, and an NVIDIA GeForce RTX 4090 GPU for computational implementation. The simulation was conducted using Python 3.13.0/Gymnasium 1.1.1 software and Pandapower 3.1.1 to simulate the real-world power grid environment.

Figure 1. IEEE 33-Bus network.

5.1. Case Setup

In this study, the paper explores a method to integrate MGs with different network structures and varying energy consumption characteristics into the ADN. MG1 and MG4 are primarily mechanical loads with relatively low-price elasticity; MG2 focuses on Thermostatically Controlled Loads (TCL) and Electric Vehicles (EVs) charging, with prominent peaks; MG3 includes Heating, Ventilation, and Air Conditioning (HVAC) and lighting, which occupy a relatively high proportion and align with PV generation. The baseline net-load and PV profiles of each MG are represented as synthetic 24 h time series with a resolution of 1 h, constructed to reflect typical daily patterns of the four MG categories and scaled according to the DER capacities in Table 2.

Table 2. DERS parameter settings.

Parameter	Value	Parameter	Value
${\lambda }^G$	500 ($/MWh)	$R^{GT}$	0.05
$e^{GT}$	0.8 (t/MWh)	${\eta }_{cha},{\eta }_{dis}$	0.95
${\lambda }^{BESS}$	150 ($/MWh)	${\lambda }^C$	100 ($/t)

To assess the effectiveness of the proposed model, a 24-h horizon is discretized into one-hour decision intervals (T = 24). In the testing stage, simulations are performed for seven consecutive daily episodes. For each episode, the same baseline load profiles are used, while the PV output is perturbed by independent uniform noise up to 10% of the hourly maximum-power-point value, thereby generating different stochastic scenarios to emulate day-to-day variability and verify the consistency of the results.

The configuration parameters of the DERs are listed in Table 2. Within the distribution network, the internal market price adjustment range is defined as [−0.2, 0.2] ($/kWh). In the MMG layer, the controllable output ranges for energy storage systems and micro gas turbines are [−300, 300] kW and [−50, 50] kW, respectively [30]. The demand-response variation is limited to [−0.2, 0.2] times the real-time load. The corresponding MADDPG algorithm parameters are provided in Table 3. The upstream-grid carbon intensity $e^{UPG}\left(t\right)$ is modeled as an hourly time series with a resolution of 1 h and units of t/MWh (based on data provided by the supporting utility). The emission factor of the micro gas turbines $e^{GT}$ is derived by considering typical GT efficiency and the lower heating value of natural gas, and is used as a time-invariant equivalent carbon-emission factor in the simulations. The external market price ${\lambda }_{EM}\left(t\right)$ adopts a three-level time-of-use structure. Specifically, ${\lambda }_{EM}\left(t\right)$ is set to 0.246 $/kWh during 0:00–7:00 and 12:00–13:00, 0.603 $/kWh at 8:00 and 20:00–23:00, and 1.007 $/kWh during 9:00–11:00 and 14:00–19:00. This time-of-use tariff serves as the baseline external price signal, around which the internal retail price adjustment ${\lambda }_{IM}$ is applied.

Table 3. Hyperparameter settings of the MADDPG algorithm.

Hyperparameters	Value	Hyperparameters	Value
Training steps	3 × 105	Number of iterations per step	24
Strategy network learning rate	4 × 10−4	$\tau$	0.01
Value network learning rate	4 × 10−4	Gamma	0.96
Hidden layer dimensions	50 and 250	Smoothing area size	1 × 105
Dynamic batch size	✓	Layer normalization	✓
Initialization of interactions	✓	Initialization of interactions	✓
Network		Structure
Strategy network		L1 → LN → GeLU → L2 → GeLU → L3 → Tanh (policy head)
Value network		L1 → LN → GeLU → L2 → GeLU → L3 (value head)

In this case study, each algorithm is trained with a single random seed over 8000 episodes, and each episode consists of 24 one-hour decision steps, yielding about ($1.9210\times {10}^5$) decision steps in total. After collecting the first 2000 steps as warm-up data, the network parameters are updated every 4 steps to complete policy training and convergence.

5.2. Result Analysis

During both training and testing, the safety layer described in Section 4.2 is used to check and correct potential violations of the device-level constraints in Equations (16)–(24), including SOC bounds, DR power limits and energy-neutrality, and GT output and ramp limits. For the test days reported in this section, the resulting dispatch trajectories were inspected and no substantive violations of these constraints were observed, indicating that device-level operational safety is effectively ensured within the experimental scope.

Figure 2 illustrates the evolution of the agents’ total reward over a 24-step episode, together with the corresponding operational behavior. The total power output and load exhibit similar profiles with a negative correlation: between 9:00 and 15:00, MGs in the ADN increase clean electricity sales and power transfers while the regional carbon intensity decreases, leading to lower carbon emissions. In the evening, solar generation decreases as the load rises, and MG-side resources such as gas turbines and energy storage systems increase their output during the peak period while reducing the overall generation cost. The ADN reward curve (Figure 3) remains relatively stable, mainly because the electricity price varies smoothly; around noon, the ADN carbon intensity is low due to cleaner upstream supply, and the ADN reward is adjusted accordingly.

Figure 2. Total reward curve.

Figure 3. ADN reward curve.

The physical state of the area is more sensitive: the frequency of charging and discharging of energy storage is large, and the carbon emissions fluctuate greatly. During the evening, carbon emissions increase with the larger change in load. It is worth noting that the SOC value in Figure 4 reflects this significant change in the system. In summary, Figure 4 show that the optimization model is robust to both economic performance and equipment longevity. The operations are aligned with carbon goals, and the system can adaptively balance power generation with demand.

Figure 4. MG reward curve.

From Figure 5, it can be seen that the SOC of the four energy storage units remains within the constraint range throughout the 24-h operation cycle, but shows significant differences. The SOC of BESS 2 stays at a high level, indicating that it mainly serves as a backup under the scheduling strategy, participating in charging and discharging less frequently. BESS 3 charges rapidly during the day when solar power output is sufficient, and gradually discharges in the evening, reflecting the typical role of peak shaving and valley filling. The SOC of BESS 4 fluctuates multiple times throughout the day, indicating that it frequently responds to system load and price signals to achieve local balance. In contrast, BESS 1 continuously discharges in the first half of the day, with its SOC gradually decreasing to the lower limit, showing that it mainly undertakes the task of peak shaving during the day.

Figure 5. Daily SOC variation curve of the BESSs.

Overall, under the coordination of the multi-agent strategy, each energy storage unit operates in a differentiated manner, avoiding the risk of exceeding limits, while sharing energy regulation tasks across time, which helps improve the system’s flexibility and low-carbon operation efficiency.

Figure 6 shows the daily output characteristics of the gas turbines GT1, GT3, and GT4. Overall, all three units operate within their technical constraints but exhibit different scheduling characteristics. GT1 quickly ramps up in the early morning and maintains a high output for several hours, demonstrating its support role during the initial load rise phase of the system. The output curve of GT3 shows a gradual increase, reaching a relatively high level in the evening, indicating that it compensates during the evening peak and when renewable output is insufficient. In contrast, GT4 maintains the most stable output, staying close to a constant level, primarily supplying the system’s base load.

Figure 6. Daily output curve of GTs.

It is worth noting that GT1 and GT3 alternately assume the marginal regulation role at different times, which helps reduce the fluctuations and startup/shutdown frequencies of individual units. This differentiated operation mode not only enhances the economic efficiency and reliability of the gas turbine group but also, by reducing redundant startups and the dependence on high-carbon power during certain periods, creates conditions for the overall low-carbon dispatch of the system.

Figure 7 reflects the carbon intensity variation characteristics of the ADN and the four MGs over a 24-h period. From the overall trend, ADN and MG3 maintain higher carbon intensity during most periods, indicating that they mainly rely on fossil energy sources such as gas turbines for power supply, contributing significantly to system carbon emissions. In contrast, MG1 and MG2 generally exhibit lower carbon intensity, with some periods even approaching neutral levels, showing that their higher share of clean energy provides significant emission reduction benefits. MG4 operates at an intermediate level, with a noticeable increase in carbon intensity during the morning and evening peak periods, indicating that its scheduling is more influenced by load and electricity price fluctuations.

Figure 7. Hourly carbon intensity distribution heatmap for each region.

It is worth noting that at night (18:00–22:00), the carbon intensity of ADN significantly rises, suggesting that when renewable energy is insufficient, the ADN bottom-up mechanism increases the system’s marginal emissions. The differences in carbon intensity distribution across different MGs during various time periods highlight the carbon emission inconsistencies caused by the unbalanced source-load-storage structure. This provides intuitive insights for the subsequent optimization based on P2P carbon quota trading and ADN clearing mechanisms.

As can be seen from Figure 8, the carbon quota trading behaviors of different MGs exhibit significant differentiation over the course of a week. MG1 and MG2 are net buyers of carbon quotas during most periods, indicating that their load rigidity or high carbon intensity requires them to purchase quotas to meet emission constraints. In contrast, MG3 frequently becomes a net seller during the day when solar power output is sufficient, reflecting its high share of clean energy and strong emission reduction advantages. MG4 alternates between buying and selling throughout the week, positioning itself as a balanced market participant.

Figure 8. Carbon quota buy-and-sell situation of each MG within one week.

Overall, under the P2P carbon trading platform, the differentiated trading roles formed by each MG based on their own resource endowment and emission levels not only alleviate the problem of uneven quota distribution but also demonstrate the effectiveness of the trading mechanism in promoting low-carbon collaboration.

Figure 9 shows that the adjustment range of ${\lambda }_{IM}$ relative to ${\lambda }_{EM}$ is small: in most periods their difference is about 0.01 $/kWh. This implies that the internal price is a fine-grained correction to the external time-of-use tariff rather than a new price structure. When the MG cluster still has usable local flexibility, the ADN raises ${\lambda }_{IM}$ slightly above ${\lambda }_{EM}$ (by about 0.01–0.02 $/kWh) to strengthen the marginal price signal in high-price hours and encourage MGs to reduce upstream purchases through local discharge and demand response. When most MGs have already increased local output and upstream purchases have fallen, ${\lambda }_{IM}$ is kept close to or slightly below ${\lambda }_{EM}$ to avoid further price increases when local resources are nearly exhausted. Thus, the internal price can be interpreted as a state-dependent refinement of the external tariff, based on the ADN’s observation of storage and demand-response conditions, to balance economic efficiency and low-carbon objectives.

Figure 9. Internal vs. external market prices (24 h).

Figure 10 reflects the changes in the profits generated from carbon quota buying and selling by each MG over the course of a week. It can be observed that MG3, with its strong renewable output, generally has positive carbon trading profits, although with significant fluctuations. This indicates that it can generate additional income by selling excess quotas during peak periods. MG1 and MG2, being long-term net buyers, have mostly negative trading profits, implying that they face greater pressure in their low-carbon transition. The profit curve of MG4 remains close to zero, with occasional small positive and negative fluctuations, reflecting its more stable and balanced role in the market.

Figure 10. Carbon trading revenue variation of each MG within one week.

The carbon trading profits are closely related to the energy structure and load characteristics of each MG, and they also confirm that the dual carbon-electricity market mechanism can create incentive effects for MGs with a high share of clean energy, thereby promoting overall system emission reductions.

Furthermore, because the upstream-grid carbon intensity $e_{\mathrm{U}\mathrm{P}\mathrm{G}}(t)$ varies significantly over time, the case-study results also reflect the qualitative influence of carbon-intensity levels on carbon trades and dispatch decisions. During hours with higher upstream carbon intensity, the internal price adjustment of the ADN becomes more active and the MGs respond by increasing local generation and demand-side adjustments, which leads to larger carbon-quota trading volumes and more pronounced redispatch. In contrast, when $e_{\mathrm{U}\mathrm{P}\mathrm{G}}\left(t\right)$ is relatively low, the system relies more on upstream supply, the incentives for local flexibility are weaker, and both trading activity and redispatch magnitudes are reduced. These patterns indicate that the proposed mechanism reacts consistently to the temporal carbon signals embedded in $e_{\mathrm{U}\mathrm{P}\mathrm{G}}\left(t\right)$, even though the calibrated carbon-intensity parameters themselves are kept fixed across scenarios.

5.3. Comparative Analysis of Low-Carbon Economic Optimal Dispatch Decisions

To systematically evaluate the carbon reduction and economic benefits of the proposed carbon–electricity coordinated optimization mechanism, this section conducts a comparative analysis of low-carbon economic decision-making from two perspectives: “algorithm comparison” and “market configuration scenarios.”

Table 4 compares the performance of four algorithms at both the MGs and ADN levels: centralized DDPG (C-DDPG), independent DDPG (IDDPG), a rule-based algorithm (RBD), and the MADDPG algorithm adopted in this paper. In terms of carbon reduction, MADDPG achieves the lowest total emissions from MGs (27.16 t), corresponding to reductions of about 4.9% and 6.0% relative to C-DDPG and RBD, and still about 2.5% lower than the best baseline IDDPG (27.85 t). In terms of economic performance, the MG operating cost under MADDPG (2568.41 $) is 6.3% lower than C-DDPG and 4.5% lower than RBD, while being only about 3.2% higher than the more cost-oriented IDDPG baseline. At the ADN level, MADDPG also yields the lowest carbon-intensity ratio (0.85), reducing it by approximately 3–5% compared with the other baselines, while achieving the highest ADN operating profit (466.82 $), which is 8.7% and 18.7% higher than C-DDPG and RBD, and still about 1.7% higher than IDDPG. Overall, these results indicate that, compared with centralized RL, independent RL, and the rule-based algorithm, the cooperative MADDPG algorithm achieves a more favorable trade-off among cost, emissions, and profit, lowering carbon intensity and improving ADN revenues while keeping MG operating costs competitive.

Table 4. Algorithm Performance on MGs and ADN.

Algorithm	C-DDPG	IDDPG	RBD	MADDPG
Total carbon emissions of MGs $\left(t\right)$	28.55	27.85	28.9	27.16
MGs operating cost ($\mathrm{\$})$	2741.12	2488.07	2689.76	2568.41
Carbon intensity ratio of the ADN	0.88	0.86	0.89	0.85
ADN operating revenue ($)	429.33	458.9	393.23	466.82

Table 5 presents an ablation study on the carbon–electricity market design. Case 1 disables horizontal P2P carbon–electricity trading among MGs while retaining ADN clearing, Case 2 keeps P2P trading but removes the ADN backstop clearing, and Case 3 corresponds to the full two-layer mechanism. When P2P is removed (Case 1), MGs exhibit higher total emissions (30.24 t) and operating cost (2901.16$), and the ADN carbon-intensity ratio deteriorates to 0.91, although the ADN obtains the highest profit (498.50 $) because more energy is cleared at the ADN level. In contrast, removing ADN clearing but retaining P2P trading (Case 2) forces MGs to self-balance using local flexibility; this yields the lowest MG emissions (17.25 t) and the best ADN carbon-intensity ratio (0.81), but MG operating cost increases sharply to 4841.01 $ and the ADN suffers a loss (−177.40 $), which is economically unattractive. Under the full mechanism (Case 3), MG operating cost is minimized (2568.41$), the ADN carbon-intensity ratio is reduced to 0.85 (about 7% lower than Case 1), and ADN profit remains high (466.82$, only slightly below Case 1 but far above Case 2). Overall, these ablations confirm that combining P2P trading with ADN clearing yields a more balanced cost–emissions–profit trade-off than either P2P-only or ADN-only market designs.

Table 5. Comparison of Market Mechanism Cases.

Case	Case 1	Case 2	Case 3
Total carbon emissions of MGs $\left(t\right)$	30.24	17.25	27.16
MGs operating cost $\left(\mathrm{\$}\right)$	2901.16	4841.01	2568.41
Carbon intensity ratio of the ADN	0.91	0.81	0.85
ADN operating revenue ($)	498.5	−177.4	466.82

6. Conclusions

This paper proposes a low-carbon collaborative optimization method for the ADN and MMG clusters. By constructing the STCIEM, the method achieves distributed, accurate carbon emission measurement and fair distribution. The designed dual carbon-electricity market mechanism supports P2P trading and ADN bottom-up clearing, effectively enhancing the market’s flexibility and fairness. Combined with the MADDPG algorithm, each MG can learn and implement differentiated scheduling strategies in a decentralized game environment, thereby balancing the system’s economic performance with low-carbon goals.

Simulation results show that this framework can coordinate energy storage for peak shaving and valley filling, gas turbine base load support, and marginal regulation, revealing the hourly carbon intensity differences across regions. Compared to traditional centralized methods, this study has clear advantages in privacy protection, carbon intensity balancing, and market liquidity, demonstrating its feasibility in complex power markets and distributed energy environments.

In summary, the innovation of this paper lies not only in the organic combination of STCIEM and the dual carbon-electricity market but also in the introduction of multi-agent reinforcement learning into carbon-electricity collaborative optimization, providing theoretical support and practical reference for future large-scale clean energy integration and carbon neutrality path exploration.