Cloud–Edge Collaborative Resource Allocation Optimization for Low-Latency Distribution Automation in Smart Grid

Abstract

Efficient distribution automation management, which integrates data perception, real-time analysis, and operational decision-making for distribution-automation services, requires low-delay, high-reliability, and coordinated processing of the massive, multi-dimensional data generated by distribution terminal units (DTUs). However, challenges arise from the limited computing resources of DTUs, communication bottlenecks in cloud-only processing, and the dynamic, uncertain nature of wireless channels and distributed energy resources, all of which complicate real-time data analysis and decision processes. To overcome these challenges, this paper proposes a 3D backpressure learning-based cloud–edge collaborative dynamic resource allocation approach. Specifically, a cloud–edge coordinated data processing architecture and a full-link latency characterization model are constructed, and the joint optimization of terminal splitting ratio, computing paradigm determination, and computing resource allocation is formulated as a stochastic optimization formulation. The long-term delay-constrained optimization is transformed into a queue stability problem via Lyapunov optimization, and a dual-phase optimization scheme is designed: (i) a 3D backpressure-aware federated deep deterministic policy gradient (FDDPG) algorithm is constructed for optimizing terminal splitting and computing paradigm selection; (ii) a Lagrangian-based optimization strategy is adopted to allocate cloud–edge computing resources. To further enhance learning efficiency and robustness, a 3D backpressure-aware mechanism is integrated into the reward construction process, a differential contribution-based federated aggregation strategy is presented to enhance collaborative training reliability, and a dual experience pool mechanism is established to enhance constraint-boundary exploration capability and increase sample efficiency. Simulation results verify that the developed approach achieves better delay performance and faster convergence behavior, providing efficient and reliable resource scheduling support for distribution automation.

1. Introduction

Efficient distribution automation management refers to a comprehensive technological framework that leverages real-time data processing and intelligent decision-making to achieve accurate perception of distribution network operating status, rapid fault isolation and restoration, and efficient interaction of sources–grid–load–storage. This framework significantly enhances the operational efficiency and reliability of distribution networks. As the core equipment for distribution automation, distribution terminal units (DTUs) deployed on-site must frequently collect real-time data on electrical quantities, equipment status, state data, and environmental conditions. These real-time, multi-dimensional data serve as the foundation for efficient distribution automation management, providing critical decision-making support for accurately perceiving network status, rapidly diagnosing and locating faults, and coordinating the optimal control of sources–grid–load–storage. With the continuous expansion of monitoring dimensions for distribution network operating status, the data generated by core equipment such as switching stations, ring main units, and distribution substations has increased dramatically, placing greater demand on DTUs for data processing [1,2]. However, traditional local processing modes are constrained by the limited computing and storage resources of DTUs, making them difficult to meet the real-time analysis needs of large-scale data. Although centralized cloud processing can leverage the powerful computing resources of distribution network control center, the simultaneous upload of massive amounts of data from numerous DTUs can easily overwhelm communication channels. Additionally, the cloud–edge transmission path is vulnerable to factors like routing congestion and electrical interference, leading to significant delays in data processing and potential security risks [3,4]. To alleviate these pressures, edge computing utilizes server clusters near DTUs for localized data processing. However, individual edge nodes still face resource limitations. When handling concurrent requests from multiple DTUs and dealing with restricted edge-to-end transmission bandwidth, these edge nodes struggle to support global optimization tasks, such as efficient interaction between system components [5].

Therefore, it is imperative to develop a cloud–edge collaborative dynamic resource allocation method to achieve efficient processing of distribution automation terminal data, thereby providing more efficient and reliable resource scheduling decision support for distribution automation. Its core lies in enhancing system performance through the joint optimization of terminal splitting ratio decisions, computing paradigm selection, and computing resource allocation [6,7]. First, computing paradigm selection determines whether data should be handled at edge nodes or processed through cloud servers, while also balancing the workload across the edge-side cluster. Second, optimizing the terminal splitting ratio involves adaptively regulating the proportion of raw data assigned to the edge layer and the cloud layer. Finally, precise computing resource allocation, achieved by rationally allocating resources such as central processing unit (CPU) and memory, can significantly enhance task processing efficiency and reduce energy consumption. However, the joint optimization of terminal splitting ratio, computing paradigm, and computing resource allocation involves complex high-dimensional decision variables and suffers from considerable uncertainty caused by the time-varying characteristics of distribution automation environments. On the one hand, wireless channel gain fluctuates drastically due to environmental interference and other factors, complicating the accurate modeling of data transmission rates. On the other hand, the stochastic power output of distributed energy resources (DERs) such as photovoltaic systems and energy storage units introduces strong randomness into terminal data acquisition volumes. These dynamic environmental factors make it extremely difficult to construct a universally applicable and accurate mathematical model. Traditional model-based optimization methods, such as convex optimization and integer programming, heavily rely on precise perception of system states (e.g., channel conditions, task arrival rates). However, under conditions of incomplete information or model mismatches, these methods struggle with low solving efficiency and may fail to converge. As a result, solutions often fail to meet the stringent low-delay and high-reliability requirements for distribution automation.

Deep reinforcement learning (DRL) has been extensively applied to cloud–edge collaborative resource allocation due to its powerful capabilities in environmental perception and decision-making. In [4], Zhao et al. proposed a differential evolution-based algorithm for cloud–edge–terminal collaborative computing resource allocation. By integrating genetic algorithms, deep learning, and differential evolution, the proposed method enabled efficient task offloading and optimized resource scheduling. In [8], Laili et al. proposed a DRL-based dynamic terminal splitting mechanism. By designing a state prediction module incorporating a long short-term memory (LSTM) network, it accurately predicted network congestion levels and edge node load states, thereby optimizing terminal splitting ratio decisions. However, the aforementioned DRL approaches exhibit limited efficiency when addressing high-dimensional state spaces and continuous action spaces and are susceptible to overfitting. The deep deterministic policy gradient (DDPG) algorithm is well-suited for sequential decision-making problems in high-dimensional continuous state and action spaces, providing precise continuous actions for computing resource allocation in distribution automation scenarios. In [9], Alam et al. proposed a DDPG-based joint optimization algorithm for computation offloading and resource allocation, effectively addressing the challenges of high-dimensional state and action spaces in these contexts, significantly improving system utility and achieving excellent convergence performance. In [10], Wu et al. developed a DDPG-based joint resource allocation and trajectory optimization algorithm for unmanned aerial vehicle-assisted mobile edge computing (MEC) systems. They modeled the problem as a Markov decision process (MDP) and used the DDPG algorithm to solve it, achieving optimized computing resource allocation and minimizing total system energy consumption while ensuring relatively fast convergence. In [11], Song et al. introduced an iterative task offloading scheme for two-user non-orthogonal multiple access (NOMA), optimizing task allocation and offloading power using DDPG. This approach effectively reduced task processing delay and achieved optimal system resource allocation. However, traditional DDPG agents face a key limitation, i.e., they cannot effectively share local experiences on a global scale. This results in low sample utilization efficiency, hindering their ability to meet the demands of optimizing computing resource allocation in complex distribution automation environments.

Federated learning (FL) offers significant advantages in distributed collaborative training. When combined with DDPG, FL can further enhance the algorithm’s optimization capability for complex resource allocation problems, enabling more efficient and robust resource allocation. In [12], Wang et al. introduced the FL-based DDPG (FDDPG) algorithm, which optimized computational offloading and resource management for unmanned aerial vehicles, achieving notable reductions in task response delay and user energy consumption. In [13], Li et al. employed an FDDPG-based joint optimization framework for computation offloading and resource allocation, reducing computational burden, alleviating core network congestion, and decreasing system overhead. In [14], He et al. proposed a two-phase FDDPG framework that effectively optimized communication resource allocation and client scheduling, resulting in higher test accuracy and lower learning delay. While these studies provide effective solutions for cloud–edge collaborative computing resource allocation, they fail to adequately address backpressure-aware mechanisms and the reliability of federated aggregation. Notably, in [15], Chen et al. proposed a novel dynamic route control (DRC) algorithm based on distributed backpressure principles. This algorithm integrates backpressure information at the edge node level and constructs a real-time density-based backpressure mechanism, overcoming the point-queue assumption in the original algorithm and significantly reducing communication costs and execution time. In [16], Wang et al. proposed a distributed communication system based on reinforcement learning and backpressure awareness. By capturing terminal queue backlog states in real-time, along with data transmission demands and processing capabilities, it dynamically regulates traffic flow. Using reinforcement learning to adapt to changing network conditions, the system alleviates congestion and reduces packet loss. However, these works focus solely on queue states at individual hierarchical levels (e.g., edge or terminal) and fail to comprehensively account for the synergistic effects of 3D backpressure across the cloud, edge, and terminal layers. As a result, they lack the ability to accurately perceive global congestion states and respond optimally within complex cloud–edge–terminal collaborative architectures. This limitation causes the overall computing resource allocation strategy to deviate from its optimal state.

Despite extensive research in cloud–edge collaborative dynamic resource allocation, current methods still face significant challenges in meeting the stringent requirements for low delay, high reliability, and global optimization in distribution automation. First, cloud–edge collaborative resource allocation requires the coordinated optimization of terminal splitting ratio decisions, computing paradigm selection, and computing resource allocation. Traditional model-driven approaches, such as convex optimization or integer programming, heavily rely on precise system state perception and modeling (e.g., channel conditions, task arrival rates). When information is incomplete, these methods often suffer from low solving efficiency or fail to converge. Second, although DDPG has been widely applied to cloud–edge collaborative resource allocation, most existing DDPG-based methods lack a backpressure-aware mechanism. This limits their ability to dynamically adjust reward functions and decisions based on real-time queue backlog across network layers. Although some studies have attempted to incorporate backpressure mechanisms into DDPG, they typically focus only on queue states at a single hierarchical level. They fail to consider the synergistic effects of 3D backpressure across the cloud, edge, and terminal layers. As a result, these methods lack the ability to accurately perceive global congestion states and respond optimally within complex cloud–edge–terminal collaborative architectures, leading to suboptimal computing resource allocation strategies. Furthermore, while integrating FL with DDPG improves distributed collaborative optimization capabilities, existing methods do not adequately assess the reliability of edge nodes participating in federated aggregation. Traditional node selection mechanisms overlook variations in nodes’ historical behaviors and real-time processing capabilities. Consequently, the cloud server cannot prioritize the reliable and contributive nodes, leading to potential degradation in aggregation effectiveness when unreliable or low-contribution nodes are involved. Finally, experience replay is a critical component of reinforcement learning. However, traditional methods store and sample experiences indiscriminately, without considering whether they satisfy system constraints. This can lead to insufficient exploration of critical constraint boundaries and low sample utilization efficiency, ultimately hindering the agent’s ability to learn the optimal policy that satisfies stringent system constraints.

To address the aforementioned challenges, this paper proposes a cloud–edge coordinated dynamic resource allocation method for efficient distribution automation. First, a cloud–edge integrated data processing model and a full-link latency model are established. Second, a cloud–edge collaborative dynamic resource allocation optimization problem is constructed. This problem aims to minimize the end-to-end queuing delay by jointly optimizing terminal splitting ratio decisions, computing paradigm selection, and computing resource allocation, subject to long-term delay constraints on terminal, edge, and cloud data queues. Subsequently, the Lyapunov optimization method is adopted to convert long-term delay constraints into a virtual queue stability problem. By designing a Lyapunov drift-plus-penalty term, the short-term decision-making process is decoupled from long-term constraints. Consequently, the original problem is decomposed into two sub-optimization problems with logical temporal sequencing. Finally, a 3D backpressure learning-based cloud–edge collaborative dynamic resource allocation algorithm is developed. In the first stage, a backpressure-aware FDDPG-based cloud–edge collaborative communication resource allocation optimization method is employed to solve the sub-problem of terminal splitting ratio and computing paradigm decisions. In the second stage, a Lagrangian algorithm-based edge–cloud computing resource allocation optimization method is utilized to solve the sub-problem of computing resource allocation for edge servers and the cloud.

As illustrated in Table 1, existing FDDPG-based studies lack cross-layer three-dimensional backpressure awareness, node-contribution-driven federated aggregation, and constraint-oriented experience replay. Although several prior works incorporate backpressure mechanisms, they only partially consider queue congestion within a single layer rather than joint cloud–edge–terminal global perception. To fill these research gaps, this paper proposes an enhanced FDDPG framework integrating 3D backpressure awareness, differential-contribution federated aggregation, and dual-experience-pool optimization.

Table 1. Comparative summary against existing FDDPG- and backpressure-aware cloud–edge resource allocation studies.

Reference	FDDPG Framework	3D Cloud–Edge–Terminal Backpressure	Differential-Contribution Federated Aggregation	Dual Experience-Pool for Constraint Exploration
Wang et al. [12]	√	⊗	⊗	⊗
Li et al. [13]	√	⊗	⊗	⊗
He et al. [14]	√	⊗	⊗	⊗
Chen et al. [15]	⊗	○ ᵃ	⊗	⊗
Wang et al. [16]	⊗	○ ᵇ	⊗	⊗
This work	√	√	√	√

√ fully considered; ⊗ not considered; ○ partially considered. ᵃ Only single-layer edge-side backpressure is adopted. ᵇ Only single-layer terminal-side backpressure is adopted.

The key contributions of this work are summarized as follows.

• A cloud–edge collaborative dynamic resource allocation architecture for efficient distribution automation is proposed. The proposed architecture integrates DTUs in the end-device layer, edge servers in the edge layer, and a distribution network control center in the cloud layer. To address resource contention and communication/computation conflicts caused by numerous concurrent DTU requests, a cloud–edge collaborative network processing model is developed. This model utilizes terminal splitting, dynamic computing paradigm selection, and coordinated resource allocation strategies to optimize data transmission efficiency, overcome edge computing bottlenecks, and enable the dynamic and efficient allocation of computational resources. By fully leveraging the parallel processing capabilities of the edge and cloud layers, the architecture significantly reduces the end-to-end processing delay of DTU service data, providing low-delay, high-reliability resource scheduling for distribution automation.
• A 3D backpressure-aware mechanism spanning cloud–edge–terminal layers is introduced. This mechanism jointly considers queue backlog states across all layers, dynamically incorporating the differences in backlog (from DTU-local-to-edge and DTU-local-to-cloud) as key penalty terms in the reward function. In this way, inter-layer congestion levels are captured, enabling dynamic optimization of terminal splitting ratios and computing paradigm selection. Additionally, precise awareness of congestion across hierarchical layers is gained and real-time responsiveness is realized, enhancing the adaptability and optimization under bursty traffic or resource bottleneck conditions. Therefore, based on this mechanism, the low-delay requirements of distribution automation are met.
• A federated aggregation method based on differential contribution evaluation is pro-posed. The actual contribution of edge servers to model performance is quantified by measuring the reward differential in cloud test scenarios before and after federated aggregation. The historical contribution of each edge server is normalized, enabling the selection of high-contribution nodes for parameter sharing during federated aggregation. This method ensures that the cloud server prioritizes aggregating model updates from reliable, high-value nodes, effectively filtering out unreliable or low-contribution nodes. In heterogeneous or potentially untrustworthy distribution automation environments, it guarantees reliable and stable federated aggregation, thereby enhancing global optimization effectiveness.
• The dual experience pool mechanism for FDDPG is enhanced. To address the limitations of traditional experience replay (e.g., indiscriminate storage, uniform sampling, insufficient exploration of constraint boundaries, and low sample utilization efficiency), experience samples are categorized based on their adherence to delaying virtual queue constraints. Separate success and failure experience pools are established, and proportional hybrid sampling from both pools is applied during network training, which forces the agent to actively explore the system’s constraint boundaries. The mechanism significantly improves sample utilization efficiency and effectively mitigates the traditional DDPG shortcomings of inadequate exploration and susceptibility to local optima in high-constraint, highly dynamic resource allocation scenarios in distribution automation.

2. Cloud–Edge Collaborative Dynamic Resource Allocation Architecture for Efficient Distribution Automation Management

As illustrated in Figure 1, a cloud–edge collaborative dynamic resource allocation architecture is developed to support efficient distribution automation, which consists of the terminal layer, the edge layer, and the cloud layer. The terminal layer is composed of DTUs deployed at key locations in the distribution network, including switching stations, ring main units, and distribution substations. Each DTU continuously monitors electrical parameters, such as voltage and current, at high frequency from corresponding measurement points (feeders, transformers, and switching devices). The collected data can be either handled by local terminals or uploaded to the edge and cloud layers for subsequent processing.

Figure 1. Cloud–edge collaborative dynamic resource allocation architecture for efficient distribution automation.

The edge layer consists of communication base stations and edge servers. Specifically, each edge server e_n serves M_n DTUs, the edge server set is represented as E = {e₁, … , e_n, … , e_N}, the DTU set is defined as $\mathrm{D}=\{d_{1,n},\dots ,d_{m,n},\dots ,d_{M_n,n}\}$, and $d_{m,n}$ denotes the m-th data acquisition terminal under e_n. The service data collected from terminals through base stations can be either processed at the corresponding edge server or transmitted to the cloud layer through the backhaul network.

The cloud layer consists of the distribution network control center, which provides powerful computing capabilities for executing in-depth processing and analyzing high-frequency service data collected from the network. This enables efficient interaction among source, grid, load, and storage, while also offering real-time operational status information of the distribution network. The terminal, edge, and cloud layers exhibit distinct characteristics in terms of data transmission and processing capabilities. Although terminal-side local processing provides fast response, its capability is restricted by limited resources and only supports lightweight preprocessing tasks. The edge layer generally possesses greater computational capacity than the terminal layer; however, each edge node is still constrained by available resources, and its performance depends jointly on the transmission capacity from terminals and its own computing capability. The cloud layer provides extensive computing resources and can handle complex computational tasks. Nevertheless, factors including long-distance data transmission paths, forwarding delays, and network congestion cause dynamic and uncertain characteristics in the cloud–edge data transmission network. This has become a major bottleneck that restricts further improvement of cloud-side data processing performance. Therefore, depending on only one layer cannot satisfy the strict requirements of distribution automation, which requires low-delay, high-reliability operations and globally optimized, computation-intensive analysis. Hence, cloud–edge collaboration becomes indispensable. The design and optimization of data-splitting mechanisms and resource allocation strategies are crucial for exploiting the advantages of cloud–edge collaboration and achieving efficient distribution automation management [17,18]. First, the edge–terminal data splitting ratio should be adaptively adjusted, with intelligent data routing strategies adopted to reduce end-to-end delay for critical services and prevent excessive congestion at edge or cloud links. Second, cloud–edge computing resources should be coordinated and allocated according to real-time requirements, ensuring that cloud computing focuses on non-real-time global tasks, whereas edge computing provides timely local responses and effectively manages service bursts and network fluctuations.

From the perspective of a complete cyber–physical distribution-grid architecture, the proposed cloud–edge–terminal framework constitutes the sensing, communication, and computing support layer connecting physical distribution equipment with grid-level operational applications [19,20]. The physical layer consists of feeders, transformers, switching devices, DERs, and other distribution-network equipment. DTUs serve as the cyber–physical interface by continuously acquiring electrical quantities and equipment-state information from these physical assets. The 5G communication network and edge servers provide low-latency data transmission, local processing and regional coordination, while the cloud control center provides system-wide data analysis and supervisory decision support Accordingly, the computational decisions considered in this paper—including terminal data-splitting ratio, computing-paradigm selection, and cloud–edge computing-resource allocation—determine where and how rapidly the acquired data are processed. These decisions therefore provide computing support for timely distribution-grid applications such as operating-state perception, fault analysis and restoration, voltage regulation, and coordinated source–grid–load–storage operation [21,22].

The total optimization horizon is denoted as $\mathcal{T}=\{1,2,\dots ,t,\dots ,T\}$, where it is partitioned into T time slots with each slot having a duration of τ₀.

At the beginning of each time slot, every edge server collects system-state information within its jurisdiction. Based on the resource-allocation optimization framework, the edge server selects the control strategy for the current slot, including decisions on terminal splitting ratio, computing paradigm selection, and computing resource allocation. These strategies are subsequently transmitted to the cloud server, where they are integrated into a global model through FL. The edge server then sends the data-splitting ratio and computing paradigm information to its associated DTUs. Each DTU collects critical electrical data, such as voltage and current, and performs local data processing or transfers the collected information to the corresponding edge server according to the received strategy. The edge server then determines the computing resource scheduling scheme based on the allocation policy to handle offloaded data, thereby minimizing end-to-end delay for terminal services.

With the increasing penetration of inverter-based DERs, smart inverters are evolving into intelligent and communicating cyber–physical devices with enhanced sensing, adaptive control, and coordinated interaction capabilities [23]. This evolution further increases the requirements for low-latency data processing and cloud–edge coordination, while cyber security, interoperability, communication standardization, and regulatory compliance become important considerations for reliable grid integration [24]. The proposed cloud–edge collaborative architecture therefore provides an enabling communication and computing foundation for the coordinated operation of increasingly intelligent distribution-grid devices.

2.1. Cloud–Edge Collaborative Data Processing Model

The cloud, edge, and terminal layers exhibit different characteristics in terms of data transmission and processing capabilities. Compared with the terminal layer, the edge layer generally possesses stronger computing resources; however, its performance remains restricted by the competition for communication and computational resources among multiple terminals, as well as the limited terminal-to-edge transmission capability and terminal-side processing capacity. Moreover, factors including routing mechanisms, extended transmission distances, and network congestion introduce dynamic fluctuations and uncertainty into the cloud–edge transmission network, which becomes a major obstacle to further improving cloud-side data processing performance. Therefore, it is necessary to establish differentiated models for local computing, edge processing, and cloud processing according to the specific characteristics of each layer, thereby constructing an effective cloud–edge collaborative processing delay model.

Define the terminal-side data-splitting ratio variable as $o_{m,n}(t)\in \left[0,1\right]$ for terminal $d_{m,n}$. $A_{m,n}\left(t\right)$ represents the volume of data acquired by $d_{m,n}$. In slot t, a portion $A_{m,n}\left(t\right)o_{m,n}\left(t\right)$ of the data is processed locally, while a portion $A_{m,n}\left(t\right)\left(1-o_{m,n}(t)\right)$ is assigned to the cloud–edge for processing. Define the computing paradigm selection variable as z_m,n,k(t) ∈ {0,1} for $d_{m,n}$, where k ∈ {1,2}, k = 1 represents selecting the edge computing paradigm and k = 2 represents selecting the cloud computing paradigm. z_m,n,k(t) = 1 indicates the selection of an appropriate computing paradigm to process the data offloaded to the cloud or edge. Each terminal can select only one computing paradigm. The computing paradigm selection constraint is expressed as

$${\displaystyle \sum _{k=1}^2{z}_{m,n,k}(t)}=1$$

(1)

2.2. Local Data Computing Model

A local computing model is formulated, in which the backlog amount of data waiting in the local processing queue is represented as $L_{m,n}^{\mathrm{d}}\left(t\right)$. Considering the significant heterogeneity in processing capabilities among different service terminals in the distribution network, the data processing process is abstracted as a model consisting of four key components: data volume, computational complexity, processing latency, and computing resources. For terminal $d_{m,n}$, the amount of data processed locally is determined by the smaller quantity between the available local computing capability and the queue backlog, which is expressed as

$$U_{m,n}^{\mathrm{d}}\left(\begin{array}{c}t\hfill \end{array}\right)=\min \left\{{\displaystyle \frac{{\tau }_0{f}_{m,n}^{\mathrm{d}}\left(\begin{array}{c}t\hfill \end{array}\right)}{{\lambda }_{m,n}(t)}},L_{m,n}^{\mathrm{d}}\left(t\right)\right\}$$

(2)

where $f_{m,n}^d(t)$ represents the terminal’s local computing resources, and λ_m,n(t) represents the computational task complexity of terminal n in time slot t, which is measured by the required CPU cycles for processing each bit of service data. Since distribution automation tasks such as power flow calculation and fault isolation are executed by iterative algorithms, the cycles consumed per bit vary with the operating condition and the required iteration times, and λ_m,n(t) is therefore modeled as a slot-varying parameter. Such a cycles-per-bit characterization is widely adopted in cloud–edge collaborative resource allocation studies [25,26], since it retains the coupling among data volume, computing resources, and processing delay while keeping the optimization problem tractable. The same treatment is applied to the edge and cloud layers in Equations (6) and (10). Upon completing data processing within a time slot, the evolution of the local computing queue backlog between adjacent slots is formulated as

$$L_{m,n}^{\mathrm{d}}\left(\begin{array}{c}t+1\hfill \end{array}\right)=L_{m,n}^{\mathrm{d}}\left(\begin{array}{c}t\hfill \end{array}\right)-U_{m,n}^{\mathrm{d}}\left(\begin{array}{c}t\hfill \end{array}\right)+A_{m,n}\left(t\right)o_{m,n}(t)$$

(3)

According to Little’s Law, the latency caused by queuing in the local data queue can be formulated as

$$S_{m,n}^{\mathrm{d}}(t)={\displaystyle \frac{L_{m,n}^{\mathrm{d}}\left(\begin{array}{c}t\hfill \end{array}\right)}{{\overline{a}}_{m,n}^{\mathrm{d}}\left(\begin{array}{c}t\hfill \end{array}\right)}}$$

(4)

where ${\overline{a}}_{m,n}^{\mathrm{d}}(t)$ denotes the mean data arrival rate generated by terminal, i.e.,

$${\overline{a}}_{m,n}^{\mathrm{d}}\left(\begin{array}{c}t\hfill \end{array}\right)={\displaystyle \frac{1}{(t-1)}}{\displaystyle {\displaystyle \sum }_{v=1}^{t-1}}{A}_{m,n}\left(\begin{array}{c}v\hfill \end{array}\right)$$

(5)

2.3. Edge Data Processing Model

The edge server maintains an independent queue for each DTU $d_{m,n}$ within its coverage area. Under the edge computing paradigm, the amount of data that can be processed by the edge server, denoted as $U_{m,n}^{\mathrm{edg}}\left(\begin{array}{c}t\hfill \end{array}\right)$, is determined by the minimum between its computing capability assigned to the queue and the current backlog of the edge data queue $d_{m,n}$, which is expressed as

$$U_{m,n}^{\mathrm{edg}}\left(\begin{array}{c}t\hfill \end{array}\right)=\min \left\{{\displaystyle \frac{{\tau }_0{f}_{m,n}^{\mathrm{edg}}\left(t\right)}{{\lambda }_{m,n}(t)}},L_{m,n}^{\mathrm{edg}}\left(t\right)\right\}$$

(6)

where $f_{m,n}^{\mathrm{edg}}\left(t\right)$ represents the computing resource allocated to e_n for processing data $d_{m,n}$, and the inter-slot evolution of the edge data queue backlog $L_{m,n}^{\mathrm{edg}}\left(t\right)$ is expressed as

$$L_{m,n}^{\mathrm{edg}}\left(t+1\right)=L_{m,n}^{\mathrm{edg}}\left(t\right)-U_{m,n}^{\mathrm{edg}}\left(t\right)+z_{m,n,1}(t)A_{m,n}\left(t\right)\left(1-o_{m,n}(t)\right)$$

(7)

where $z_{m,n,1}(t)A_{m,n}\left(t\right)\left(1-o_{m,n}(t)\right)$ denotes the data volume transferred from terminal $d_{m,n}$ to edge server e_n for execution during time slot t.

The queuing delay of terminal $d_{m,n}$ in its edge data queue at edge server e_n is expressed as

$$S_{m,n}^{\mathrm{edg}}\left(t\right)={\displaystyle \frac{L_{m,n}^{\mathrm{edg}}\left(t\right)}{{\overline{a}}_{m,n}^{\mathrm{edg}}\left(t\right)}}$$

(8)

where ${\overline{a}}_{m,n}^{\mathrm{edg}}\left(t\right)$ denotes the mean rate of incoming data at the edge side, i.e.,

$${\overline{a}}_{m,n}^{\mathrm{edg}}\left(t\right)={\displaystyle \frac{1}{(t-1)}}{\displaystyle {\displaystyle \sum }_{v=1}^{t-1}}{z}_{m,n,1}(v)A_{m,n}\left(v\right)\left(1-o_{m,n}(v)\right)$$

(9)

2.4. Cloud Data Processing Model

The cloud layer provides considerable storage capacity and computational resources, allowing it to perform power data analysis and management for information uploaded from edge devices and terminal devices through approaches such as data mining and deep learning. Under the cloud computing paradigm, the volume of data $U_{m,n}^{\mathrm{clo}}\left(\begin{array}{c}t\hfill \end{array}\right)$ that the distribution network control center can process is determined by the minimum value between the data amount delivered through the cloud–edge channel and the current backlog of the cloud data queue, which is expressed as

$$U_{m,n}^{\mathrm{clo}}\left(\begin{array}{c}t\hfill \end{array}\right)=\min \left\{{\displaystyle \frac{{\tau }_0{f}_{m,n}^{\mathrm{clo}}\left(t\right)}{{\lambda }_{m,n}(t)}},L_{m,n}^{\mathrm{clo}}\left(\begin{array}{c}t\hfill \end{array}\right)\right\}$$

(10)

where $f_{m,n}^{\mathrm{clo}}\left(t\right)$ denotes the computing capability provided by the cloud layer to handle the data generated from terminal $d_{m,n}$.

The inter-slot evolution of the cloud data queue backlog $L^{\mathrm{clo}}\left(\begin{array}{c}t\hfill \end{array}\right)$ is expressed as

$$L_{m,n}^{\mathrm{clo}}\left(\begin{array}{c}t+1\hfill \end{array}\right)=L_{m,n}^{\mathrm{clo}}\left(\begin{array}{c}t\hfill \end{array}\right)-U_{m,n}^{\mathrm{clo}}\left(\begin{array}{c}t\hfill \end{array}\right)+z_{m,n,2}(t)A_{m,n}\left(t\right)\left(1-o_{m,n}(t)\right)$$

(11)

where the third term represents the data amount transferred from edge server e_n to the cloud layer for processing during time slot t.

The queuing latency of the cloud data queue is formulated as

$$S_{m,n}^{\mathrm{clo}}\left(t\right)={\displaystyle \frac{L_{m,n}^{\mathrm{clo}}\left(t\right)}{{\overline{a}}_{m,n}^{\mathrm{clo}}\left(t\right)}}$$

(12)

where ${\overline{a}}_{m,n}^{\mathrm{clo}}\left(t\right)$ denotes the mean incoming data rate at the cloud side, i.e.,

$${\overline{a}}_{m,n}^{\mathrm{clo}}\left(t\right)={\displaystyle \frac{1}{(t-1)}}{\displaystyle {\displaystyle \sum }_{v=1}^{t-1}}{z}_{m,n,2}(v)A_{m,n}\left(v\right)\left(1-o_{m,n}(v)\right)$$

(13)

2.5. Cloud–Edge Collaborative Data Processing Delay Model

Taking into account the heterogeneous characteristics of local computing, edge processing, and cloud processing, the data of terminal d_m,n is simultaneously executed at the edge server and in the cloud. Therefore, the end-to-end queuing delay S_m,n(t) of terminal d_m,n under cloud–edge collaboration is calculated by combining the local computing queueing delay with the larger value between the edge and cloud processing queuing delays, which is expressed as

$$S_{m,n}(t)=S_{m,n}^{\mathrm{d}}(t)+\max \left\{\left.S_{m,n}^{\mathrm{edg}}(t)+S_{m,n}^{\mathrm{clo}}(t)\right|\forall d_{m.n}\in \mathcal{D}\right\}$$

(14)

In the context of efficient distribution automation, the increase in service data volume can result in significant queuing delays within the service data queues, which may adversely affect the accuracy of operational decisions in the distribution network. Therefore, to achieve effective distribution automation management through cloud–edge collaboration, it is necessary to incorporate queuing delay constraints for the local data queue, edge data queue, and cloud data queue, which are formulated as

$$\underset{T\to \infty }{\lim }{\displaystyle \frac{1}{T}}{\displaystyle \sum _{t=1}^T{S}_{m,n}^{\mathrm{d}}(t)<S_{\max }^{\mathrm{d}}},\forall d_{m,n}$$

(15)

$$\underset{T\to \infty }{\lim }{\displaystyle \frac{1}{T}}{\displaystyle \sum _{t=1}^T{S}_{m,n}^{\mathrm{edg}}\left(t\right)<S_{\max }^{\mathrm{edg}}},\forall d_{m,n}$$

(16)

$$\underset{T\to \infty }{\lim }{\displaystyle \frac{1}{T}}{\displaystyle \sum _{t=1}^T{S}_{m,n}^{\mathrm{clo}}\left(t\right)<S_{\max }^{\mathrm{clo}}},\forall d_{m,n}$$

(17)

where $S_{\max }^{\mathrm{d}}$, $S_{\max }^{\mathrm{edg}}$, and $S_{\max }^{\mathrm{clo}}$ represent the threshold values for local queuing delay, edge queuing delay, and cloud queuing delay, respectively.

3. Problem Formulation

3.1. Optimization Problem Modeling

Distribution automation imposes strict requirements on data processing delays. To address this, we aim to minimize the end-to-end queuing latency in distribution automation scenarios through cloud–edge collaborative resource optimization. Specifically, the terminal splitting ratio determines how data is distributed among local terminals, edge servers, and the cloud center, while the selection of the computing paradigm determines whether data is processed at the edge or in the cloud. An appropriate task splitting and computing paradigm strategy can achieve load balancing across different layers, avoid excessive data accumulation at individual nodes, and improve the coordination of queue backlogs among layers, thereby reducing the overall end-to-end delay [27,28]. Additionally, the dynamic allocation of computing resources between edge servers and the cloud enables better adaptation to data processing requirements, reduces unnecessary resource consumption, improves resource utilization, accelerates task execution, and further decreases queuing delay [29,30]. Accordingly, through cloud–edge collaborative optimization, we formulate the problem of minimizing the global end-to-end queuing delay in a time slot, which is expressed as

$$\begin{array}{ll}\mathrm{P}1:\hfill & \underset{\left\{o_{m,n}(t),z_{m,n,k}(t),f_{m,n}^{\mathrm{edg}}\left(t\right),f_{m,n}^{\mathrm{clo}}\left(t\right)\right\}}{\min }{\displaystyle \frac{1}{T{\displaystyle \sum _{n=1}^N{M}_n}}}{\displaystyle \sum _{n=1}^N{\displaystyle \sum _{m=1}^{M_n}{\displaystyle \sum _{t=1}^T{S}_{m,n}(t)}}}\\ \mathrm{s}.\mathrm{t}.\hfill & C_1:o_{m,n}(t)\in \left[0,1\right],\forall d_{m,n},\forall t\hfill \\ \hfill & C_2:z_{m,n,k}(t)\in \left\{0,1\right\},\forall d_{m,n},\forall t\hfill \\ \hfill & C_3:{\displaystyle \sum _{k=1}^2{z}_{m,n,k}(t)}=1\hfill \\ \hfill & C_4:{\displaystyle \sum _{m=1}^N{f}_{m,n}^{\mathrm{edg}}\left(t\right)}\le f_{n,\max }^{\mathrm{edg}}\left(t\right),\forall d_{m,n},\forall t\hfill \\ \hfill & C_5:{\displaystyle \sum _{n=1}^N{\displaystyle \sum _{m=1}^{M_n}{f}_{m,n}^{\mathrm{clo}}\left(t\right)}\le f_{\max }^{\mathrm{clo}}\left(t\right),\forall d_{m,n},\forall t}\hfill \\ \hfill & C_6:\left(15\right)~(17)\hfill \end{array}$$

(18)

where C₁, C₂, C₃ represent the constraints related to terminal splitting ratio and computing paradigm selection. C₄ represents the edge server computing resource constraint, where $f_{n,\max }^{\mathrm{edg}}\left(t\right)$ indicates the maximum computing capability available at the edge server. e_n. C₅ represents the cloud computing resource constraint, where $f_{\max }^{\mathrm{clo}}\left(t\right)$ denotes the maximum computing capability provided by the cloud. C₆ corresponds to the long-term queuing delay requirement imposed on data queues across the terminal, edge, and cloud layers.

3.2. Problem Transformation

Because short-term terminal splitting ratios, computing paradigm selection, and the allocation of computing resources are jointly governed by a long-term queuing-delay requirement, directly solving the original optimization problem P1 is challenging. To overcome this difficulty, virtual queues are constructed according to Lyapunov optimization theory, thereby converting constraint C₆ into a queue-stability condition. More specifically, the long-term queuing-delay requirements for the terminal, edge, and cloud are represented by $H_{m,n}^{\mathrm{d}}\left(t\right)$, $H_{m,n}^{\mathrm{edg}}\left(t\right)$ and $H_{m,n}^{\mathrm{clo}}\left(t\right)$, respectively, and are formulated as

$$\begin{array}{l}{H}_{m,n}^{\mathrm{d}}\left(t+1\right)={\left[H_{m,n}^{\mathrm{d}}\left(t\right)+S_{m,n}^{\mathrm{d}}(t)-S_{\max }^{\mathrm{d}}\right]}^{+}\hfill \\ H_{m,n}^{\mathrm{edg}}\left(t+1\right)={\left[H_{m,n}^{\mathrm{edg}}\left(t\right)+S_{m,n}^{\mathrm{edg}}\left(t\right)-S_{\max }^{\mathrm{edg}}\right]}^{+}\hfill \\ H_{m,n}^{\mathrm{clo}}\left(t+1\right)={\left[H_{m,n}^{\mathrm{clo}}\left(t\right)+S_{m,n}^{\mathrm{clo}}\left(t\right)-S_{\max }^{\mathrm{clo}}\right]}^{+}\hfill \end{array}$$

(19)

where [⋅]⁺ ensures that the queue deficits remain non-negative.

The virtual queue backlogs $H_{m,n}^{\mathrm{d}}\left(t\right)$, $H_{m,n}^{\mathrm{edg}}\left(t\right)$ and $H_{m,n}^{\mathrm{clo}}\left(t\right)$ respectively capture the deviations between the queuing delays of the terminal, edge, and cloud data queues and their corresponding thresholds. According to [31], when all three virtual queues are stabilized, constraint C₆ is guaranteed to be satisfied. In this way, the short-term decisions are decoupled from the long-term constraint, and problem P1 can be transformed as

$$\begin{array}{l}\mathrm{P}2:\underset{\left\{o_{m,n}(t),z_{m,n,k}(t),f_{m,n}^{\mathrm{edg}}\left(t\right),f_{m,n}^{\mathrm{clo}}\left(t\right)\right\}}{\min }\psi (t)={\displaystyle \frac{1}{{\displaystyle \sum _{n=1}^N{M}_n}}}{\displaystyle {\displaystyle \sum _{n=1}^N}{\psi }_n(t)}={\displaystyle \frac{1}{{\displaystyle \sum _{n=1}^N{M}_n}}}{\displaystyle {\displaystyle \sum _{n=1}^N}{\displaystyle \sum _{m=1}^{M_n}\left[S_{m,n}(t)\right.}}\hfill \\ \left.+{\varpi }_D\left[H_{m,n}^d\left(t\right)S_{m,n}^{\mathrm{d}}\left(t\right)\right]+{\varpi }_E\left[H_{m,n}^{\mathrm{edg}}\left(t\right)S_{m,n}^{\mathrm{edg}}\left(t\right)\right]+{\varpi }_C\left[H_{m,n}^{\mathrm{clo}}\left(t\right)S_{m,n}^{\mathrm{clo}}\left(t\right)\right]\right]\hfill \\ \mathrm{s}.\mathrm{t}.C_1:o_{m,n}(t)\in \left[0,1\right],\forall d_{m,n},\forall t\hfill \\ C_2:z_{m,n,k}(t)\in \left\{0,1\right\},\forall d_{m,n},\forall t\hfill \\ C_3:{\displaystyle \sum _{k=1}^2{z}_{m,n,k}(t)}=1\hfill \\ C_4:{\displaystyle \sum _{m=1}^N{f}_{m,n}^{\mathrm{edg}}\left(t\right)}\le f_{n,\max }^{\mathrm{edg}}\left(t\right),\forall d_{m,n},\forall t\hfill \\ C_5:{\displaystyle \sum _{n=1}^N{\displaystyle \sum _{m=1}^{M_n}{f}_{m,n}^{\mathrm{clo}}\left(t\right)}\le f_{\max }^{\mathrm{clo}}\left(t\right),\forall d_{m,n},\forall t}\hfill \end{array}$$

(20)

where ϖ_D, ϖ_E and ϖ_C denote the weighting factors assigned to the virtual queues $H_{m,n}^{\mathrm{d}}\left(t\right)$, $H_{m,n}^{\mathrm{edg}}\left(t\right)$ and $H_{m,n}^{\mathrm{clo}}\left(t\right)$, respectively, which are introduced to achieve a balance between reducing virtual queue backlogs and minimizing end-to-end queuing delay. $H_{m,n}^{\mathrm{d}}\left(t\right)S_{m,n}^d\left(t\right)$, $H_{m,n}^{\mathrm{edg}}\left(t\right)S_{m,n}^{\mathrm{edg}}\left(t\right)$ and $H_{m,n}^{\mathrm{clo}}\left(t\right)S_{m,n}^{\mathrm{clo}}\left(t\right)$ denote the stability of the virtual queue. In each time slot, the cloud–edge resource interaction decisions are made by considering both the original optimization objective and their effect on the virtual queue. This approach guarantees that while aiming for short-term reductions in end-to-end delay, the system also maintains long-term stability, preventing performance degradation from inappropriate short-term decisions. The coefficients ϖ_D, ϖ_E, and ϖ_C represent the relative priorities assigned to the terminal-, edge-, and cloud-side virtual-queue deficits. They are heuristic priority coefficients rather than theoretically optimal parameters. Since the terminal layer has the strictest latency requirement and the most limited computing capability, it is assigned the highest weight. The edge layer receives an intermediate weight because its resources are shared among multiple DTUs, whereas the cloud layer has the largest computing capability and is assigned the lowest weight. Accordingly, the normalized setting (ϖ_D,ϖ_E,ϖ_C) = (0.5,0.3,0.2) is adopted in the simulations.

4. 3D Backpressure Learning-Based Cloud–Edge Collaborative Dynamic Resource Allocation Algorithm

In the cloud–edge collaborative computing architecture, low-delay data processing for distribution automation requires coordinated interactions among terminals, edge servers, and the cloud center, involving the joint optimization of terminal splitting ratio, computing paradigm selection, and computing resource allocation [32]. The resulting optimization problem P2 is a mixed-integer NP-hard problem. To improve computational tractability, P2 is decomposed into two sequential subproblems: SP1 determines the terminal splitting ratio and computing paradigm through the FDDPG-based method, whereas SP2 allocates the corresponding edge and cloud computing resources. Since SP1 is solved using a learning-based method, this decomposition provides a tractable solution framework but does not establish global optimality for the original mixed-integer problem P2. The resulting optimization problem P2 is a mixed-integer NP-hard problem. Since the terminal splitting-ratio and computing-paradigm decisions directly determine the workloads assigned to the local, edge, and cloud processing paths, P2 is decomposed into two sequentially coupled subproblems. In each time slot t, SP1 first determines the terminal splitting ratios and computing paradigms according to the current terminal–edge–cloud system state. These decisions determine the workload entering the edge and cloud layers and therefore define the resource demands subsequently handled by SP2. Given the decisions obtained from SP1, SP2 then allocates the available edge and cloud computing resources to the corresponding workloads. The resulting processing rates update the terminal, edge, cloud, and virtual queues, and the updated queue states are used as the input state of SP1 in time slot t + 1. Therefore, SP1 and SP2 are solved sequentially within each time slot but remain dynamically coupled across successive slots through the queue evolution. Based on this decomposition, a 3D backpressure learning-based cloud–edge collaborative dynamic resource allocation algorithm is developed, as illustrated in Figure 2 and detailed in Algorithm 1. Specifically, SP1 is solved using the 3D backpressure-aware FDDPG method, while SP2 is solved using the Lagrangian-based cloud–edge computing resource-allocation method.

Figure 2. Principle of 3D backpressure learning-based cloud–edge collaborative dynamic resource allocation algorithm.

Although the FDDPG agent does not know the exact computing-resource allocation obtained by the subsequent Lagrangian stage when making the current splitting and paradigm decisions, the effects of workload arrivals and resource insufficiency are directly reflected in the queue backlogs of the terminal, edge, and cloud layers. Since these backlog states are included in the FDDPG state space, the agent can dynamically adjust the splitting ratios and computing paradigms according to the observed congestion level. Therefore, when bursty traffic temporarily increases the workload beyond the available processing capability, the resulting queue accumulation is fed back to the FDDPG agent in the subsequent decision, enabling adaptive workload redistribution and preventing persistent queue bottlenecks. This queue-feedback property is further verified by the traffic-load experiment in Section 5, where the delay of the proposed algorithm grows more slowly than those of the baselines as the data arrival volume increases,.

Algorithm 1 3D backpressure learning-based cloud–edge collaborative dynamic resource allocation algorithm.

Input: Number of time slots and slot duration; DTU and edge-server sets; initial data-queue and virtual-queue states; FDDPG learning parameters; experience-pool parameters; and system parameters defined in Section 2, Section 3 and Section 4. Output: Terminal splitting-ratio decisions, computing-paradigm decisions, and edge/cloud computing-resource allocations.     Initialize the actor, critic, target actor, and target critic networks for each edge server     Initialize the data queues, virtual queues, and success/failure experience pools according to their prescribed initial states     for each time slot t = 1, …, T do     //Stage 1: 3D backpressure-aware FDDPG-based splitting ratio and computing paradigm decision         for each edge server do in parallel           Observe the cloud–edge–terminal state according to (22).           Generate the action according to (25) and (26).           Execute the action and evaluate the reward according to (24).           Classify the resulting experience into the success or failure experience pool according to the virtual-queue-deficit criterion defined in Section 4.1.2. Sample a mini-batch from the two experience pools according to (27).           Update the critic network according to (28)–(30).           Update the actor network according to (31)–(33).           Upload the updated local model parameters to the cloud. end for           Evaluate and normalize the historical contribution scores of the edge servers according to (34).           Perform differential federated aggregation according to (35) and (36). Broadcast the aggregated global model and update the target networks according to (37) and (38).         //Stage 2: Lagrangian-based edge–cloud computing-resource allocation           Construct the edge- and cloud-side computing-resource-allocation subproblems according to (39) and (40).           Solve the corresponding resource-allocation subproblems using the Lagrangian method.           Update the data queues and virtual queues for the next time slot. end for

4.1. 3D Backpressure-Aware FDDPG-Based Cloud–Edge Collaborative Communication Resource Allocation Method

The subproblem for optimizing terminal splitting ratios and computation paradigms at the edge server side is formulated as

$$\begin{array}{ll} \mathrm{SP}1:\hfill & \underset{\left\{o_{m,n}(t),z_{m,n,k}(t)\right\}}{\min }{\psi }_n(t)\\ \mathrm{s}.\mathrm{t}.\hfill & C_1:o_{m,n}(t)\in \left[0,1\right],\forall d_{m,n},\forall t\hfill \\ \hfill & C_2:z_{m,n,k}(t)\in \left\{0,1\right\},\forall d_{m,n},\forall t\hfill \\ \hfill & C_3:{\displaystyle \sum _{k=1}^2{z}_{m,n,k}(t)}=1\hfill \end{array}$$

(21)

For SP1, this paper employs the DDPG method to solve the problem. DDPG combines the efficiency of policy gradient methods with the convergence properties of Q-learning, making it well-suited for handling high-dimensional action spaces, and is widely used in resource allocation optimization. However, traditional DDPG has limitations, including restricted global sharing of local experiences and the absence of a backpressure-aware mechanism. As a result, it struggles to adjust decisions quickly under bursty traffic or resource bottlenecks, leading to insufficient exploration efficiency in complex distribution environments and an increased likelihood of getting stuck in local optima. Additionally, existing federated aggregation methods typically use weighted averaging based on local sample sizes, which fails to account for the actual contributions of different DTUs under varying operational conditions. These methods also lack effective mechanisms for evaluating performance improvements in the aggregated model, thereby limiting global optimization capability and stability.

Therefore, this paper proposes a 3D backpressure-aware FDDPG-based cloud–edge collaborative communication resource allocation method. First, a 3D backpressure-aware mechanism is developed by incorporating backpressure terms into the reward function, enhancing the policy’s sensitivity to multi-layer queue pressure balancing. Second, a dual experience replay mechanism in FDDPG is utilized to separately store experiences under different deficit variations, with proportional sampling during training. This approach improves sample utilization and exploration efficiency while avoiding local optima in dynamic resource allocation. Finally, a cloud-side federated aggregation method based on differential contribution evaluation is introduced, where the cloud evaluates the historical contributions of edge servers’ model parameters, assigns corresponding scores, and performs differential federated aggregation, thereby improving cross-terminal policy consistency and global performance.

4.1.1. MDP Model Construction for Cloud–Edge Collaborative Communication Resource Allocation

To further describe the dynamic optimization procedure in SP1, we formulate it as an MDP. Within this formulation, the state space comprises the operating states of terminals, edge servers, and the cloud, whereas the action space includes terminal splitting ratios together with computing paradigm choices for the present time slot. In contrast to standard MDPs, the reward function introduces a 3D backpressure term as an optimization-oriented signal, allowing the model to adaptively identify load imbalance throughout the multi-layer architecture.

(1)

State space

For an edge server e_n, the state space at time slot t contains information from the cloud, edge, and terminal layers. At the terminal layer, the state comprises the terminal queue backlog, the average data arrival rate at the terminal, and the virtual queue associated with the long-term terminal queuing delay constraint. At the edge layer, it contains the edge server queue backlog, the mean data arrival rate at the edge, and the virtual queue corresponding to the long-term edge queuing delay constraint. At the cloud layer, the state incorporates the cloud server queue backlog, the average data arrival rate at the cloud, and the virtual queue related to the long-term cloud queuing delay constraint. The state space is expressed as

$$\begin{array}{ll}{S}_n(t)=\hfill & \left\{L_{m,n}^{\mathrm{d}}\left(t\right),L_{m,n}^{\mathrm{edg}}\left(t\right),L_{m,n}^{\mathrm{clo}}\left(t\right),{\overline{a}}_{m,n}^{\mathrm{d}}\left(t\right),{\overline{a}}_{m,n}^{\mathrm{edg}}\left(t\right),\right.\hfill \\ \hfill & \left.{\overline{a}}_{m,n}^{\mathrm{clo}}\left(t\right),H_{m,n}^{\mathrm{d}}\left(t\right),H_{m,n}^{\mathrm{edg}}\left(t\right),H_{m,n}^{\mathrm{clo}}\left(t\right)\left|\forall d_{m,n}\right.\in {\mathcal{D}}_n\right\}\hfill \end{array}$$

(22)

(2)

Action space

For an edge server e_n, the action space contains the terminal splitting-ratio choices for its associated DTUs, together with the computing paradigm decisions, and is formulated as

$$V_n(t)=\left\{o_{m,n}\left(t\right),z_{m,n,k}\left(t\right)\left|\forall d_{m,n}\right.\in {\mathcal{D}}_n\right\}$$

(23)

(3)

Reward space

In designing the reward function for edge server e_n, the 3D backpressure mechanism is incorporated to account for the backlog differences between DTUs and the edge, as well as between DTUs and the cloud. This design enables the model to dynamically capture load imbalances across the multi-layer architecture. Accordingly, the proposed 3D backpressure mechanism uses the terminal queue as a common reference to simultaneously evaluate the congestion differences between the terminal and the edge and between the terminal and the cloud. The two pressure components are directly coupled with the corresponding edge- and cloud-processing decisions. The reward function is therefore formulated as

$$\begin{array}{ll}{r}_n(t)=\hfill & \vartheta {\displaystyle \sum _{m=1}^{M_n}}\left[\left(L_{m,n}^d(t)-L_{m,n}^{\mathit{edg}}(t)\right)\left(1-o_{m,n}(t)\right)z_{m,n,1}(t)\right.\\ & \left.+\left(L_{m,n}^d(t)-L_{m,n}^{\mathit{clo}}(t)\right)\left(1-o_{m,n}(t)\right)z_{m,n,2}(t)\right]-{\psi }_n(t)\hfill \end{array}$$

(24)

where ϑ represents the weight of the 3D backpressure term. Different from conventional hop-by-hop multi-layer backpressure, which generally constructs pressure differences between adjacent processing layers and applies them to successive forwarding decisions, the proposed mechanism simultaneously evaluates the terminal–edge and terminal–cloud processing paths using the terminal queue as a common reference. The two pressure differences are directly coupled with the corresponding computing-paradigm decisions in (24), enabling the FDDPG policy to jointly determine the splitting ratio and computing-paradigm decisions according to the congestion conditions of the two downstream processing paths.

The positive sign of the backpressure term does not reward queue accumulation itself. When the terminal backlog is larger than the edge backlog, $L_{m,n}^d(t)-L_{m,n}^{\mathit{edq}}(t)>0$, assigning more offloaded workload to edge processing increases the corresponding reward term. Conversely, when the edge queue is more congested than the terminal queue, the pressure difference becomes negative and further edge assignment is discouraged. The same mechanism applies to the cloud-processing path. Therefore, the backpressure term rewards workload-routing decisions that alleviate inter-layer congestion, while −ψ_n(t) accounts for the end-to-end queuing-delay and virtual-queue costs.

4.1.2. Training Method for the Edge Learning Model

To solve the aforementioned MDP problem, a DDPG-based learning model is deployed at the edge server to achieve dynamic optimization of terminal splitting ratios and computing paradigms. Each edge server first determines an action according to the presently observed state, records the resulting execution result as an experience, and stores it in the corresponding experience pool. Experiences are then proportionally sampled from both pools to train the edge-side model parameters. After training, the newly optimized model parameters are transmitted to the cloud, where a differential contribution evaluation is performed to aggregate the parameters into a unified global model. The aggregated model is subsequently distributed back to the edge servers, thereby enabling joint optimization across nodes.

At edge server e_n, four deep neural networks are deployed: the main actor network ${\eta }_n^{\mathrm{main}}\left(S_n(t);{\omega }_n^{\eta ,\mathrm{main}}\right)$ with parameters ${\omega }_n^{\eta ,\mathrm{main}}$, the main critic network $Q_n^{\mathrm{main}}\left(S_n(t);V_n(t);{\omega }_n^{Q,\mathrm{main}}\right)$ with parameters ${\omega }_n^{Q,\mathrm{main}}$, the target actor network ${\eta }_n^{\mathrm{target}}\left(S_n(t);{\omega }_n^{\eta ,\mathrm{target}}\right)$ with parameters ${\omega }_n^{\eta ,\mathrm{target}}$, and the target critic network $Q_n^{\mathrm{target}}\left(S_n(t);V_n(t);{\omega }_n^{Q,\mathrm{target}}\right)$ with parameters ${\omega }_n^{Q,\mathrm{target}}$. The parameter update process of the edge learning model is described as follows.

(1) Action Policy Formulation: To enhance the exploration capability of the network, 3D backpressure-aware adaptive Gaussian noise is added to the output of the actor network before action selection. This design increases the probability of sampling non-optimal actions, thereby improving exploration efficiency. The final action is expressed as

$$V_n\left(t\right)\leftarrow {\eta }_n^{\mathrm{main}}\left(S_n(t);{\omega }_n^{\eta ,\mathrm{main}}\right)+{\beta }_n\left(t\right)$$

(25)

where β_n(t) denotes the 3D-backpressure-aware adaptive Gaussian noise, defined as

$${\beta }_n\left(t\right)\sim \mathbb{N}\left(0,\chi \exp \left(\kappa {\displaystyle {\displaystyle \sum _{m=1}^{M_n}}\left(L_{m,n}^{\mathrm{d}}\left(t\right)-L_{m,n}^{\mathrm{edg}}\left(t\right)\right)}+{\displaystyle {\displaystyle \sum _{m=1}^{M_n}}\left(L_{m,n}^{\mathrm{d}}\left(t\right)-L_{m,n}^{\mathrm{clo}}\left(t\right)\right)}\right)\right)$$

(26)

where χ and κ are tuning coefficients. As the backpressure increases, the noise level correspondingly grows, thereby encouraging the network to perform more extensive exploration.

(2) Experience Pool Partitioning: Each experience is categorized based on whether it satisfies the virtual queue delay constraint. Specifically, if the total deficit continues to increase, or if the deficit of any terminal with a positive backlog further increases, the experience is regarded as a failure and stored in the failure experience pool ${\Omega }_n^{\mathit{failure}}$. Otherwise, it is considered a successful experience and stored in the success experience pool ${\Omega }_n^{\mathit{success}}$. During model training, experiences are proportionally sampled from both pools, and the sampled training experience set Ε_n is expressed as

$${{\rm E}}_n=\mathrm{Sample}({\Omega }_n^{\mathrm{success}},\left(1-\nu \right)K)\cup \mathrm{Sample}({\Omega }_n^{\mathrm{failure}},\nu K)$$

(27)

where the function Sample(Ω,k) represents sampling k experiences from the experience pool Ω. K represents the mini-batch size, i.e., the number of experiences drawn from the pools for each parameter update. The parameter ν is the sampling ratio used to adjust the proportion of different types of experiences in each mini-batch. When the edge server is currently in a failure state, ν is set to ν^failure; otherwise, when it is in a success state, ν is set to ν^success. It should be noted that ν determines the mini-batch composition independently of the actual numbers of experiences stored in the two pools. Therefore, even when failure experiences are more common during the early training stage, their proportion in each gradient update is bounded by ν^failure rather than following the failure-pool proportion directly. Moreover, Equation (27) is activated only when both pools contain sufficient experiences to satisfy the required sampling numbers; before this condition is met, conventional replay over the accumulated experiences is used. This prevents both excessive sampling of early failure experiences and repeated sampling of a small number of successful experiences, thereby maintaining stable policy updates.

(3) Policy Performance Evaluation: The target critic network mainly estimates the Q-value associated with the present state–action pair. For the k-th experience, the target Q-value $y_n^{\mathrm{k}}(t)$ is formulated as

$$y_n^k(t)=r_n^k\left(t\right)+\gamma Q_n^{\mathrm{target}}\left(S_n^k(t+1);V_n^k(t+1);{\omega }_n^{Q,\mathrm{target}}\right)$$

(28)

where γ represents the discount factor.

The proposed algorithm uses the temporal-difference (TD) error to train the critic network. This TD error measures the gap between the presently predicted Q-value and its target obtained from the next time slot, and is formulated as

$${\delta }_n^k(t)=y_n^k(t)-Q_n^{\mathrm{main}}\left(S_n^k(t);V_n^k(t);{\omega }_n^{Q,\mathrm{main}}\right)$$

(29)

The main critic network optimizes its parameters using the mean squared error (MSE) computed from the TD error. Its loss function is formulated as

$${\delta }_n^k(t)=y_n^k(t)-Q_n^{\mathrm{main}}\left(S_n^k(t);V_n^k(t);{\omega }_n^{Q,\mathrm{main}}\right)$$

(30)

The policy function network updates its parameters in the direction of the gradient using the policy gradient signals provided by the critic. The policy performance objective function is expressed as

$$J({\omega }_n^{\eta ,\mathrm{main}})={\displaystyle \frac{1}{K}}{\displaystyle \sum _{k=1}{Q}_n^{\mathrm{main}}\left(S_n^k(t);V_n^k(t);{\omega }_n^{Q,\mathrm{main}}\right)}$$

(31)

The proposed algorithm updates the local network parameters through gradient descent, as formulated below:

$${\omega }_n^{Q,\mathrm{main}}\leftarrow {\omega }_n^{Q,\mathrm{main}}-{\mu }^Q{\nabla }_{{\omega }_n^Q}{\mathit{Loss}}_n({\omega }_n^{Q,\mathrm{main}})$$

(32)

$${\omega }_n^{\eta ,\mathrm{main}}\leftarrow {\omega }_n^{\eta ,\mathrm{main}}-{\mu }^{\eta }{\nabla }_{{\omega }_n^{\eta }}J({\omega }_n^{\eta ,\mathrm{main}})$$

(33)

where ${\nabla }_{{\omega }_n^Q}{\mathit{Loss}}_n({\omega }_n^Q)$ denotes the loss-function gradient Loss_n calculated relative to the main critic network parameters ${\omega }_n^Q$. ${\mu }^Q$ and ${\mu }^{\eta }$ represent the learning rates of the critic network and the policy function network, respectively.

During the proposed edge learning model’s training stage, a 3D backpressure-aware mechanism is integrated. This 3D backpressure strategy efficiently identifies traffic-load disparities throughout the multi-layer architecture. More specifically, queue-backlog gaps from DTUs to the edge and from DTUs to the cloud are incorporated into the reward function as adaptive feedback signals. This design enhances the model’s awareness of global task load conditions and significantly improves the effectiveness of resource allocation optimization in cloud–edge collaborative environment.

4.1.3. Differential Contribution Evaluation-Based Cloud-Side Federated Aggregation Method

In this paper, a federated learning method with differential contribution evaluation is adopted to update the network parameters of edge servers, providing global guidance for the edge servers and adjusting their strategies to improve the overall return. After updating the network parameters, each edge server uploads its network parameters ${\omega }_n^{Q,\mathrm{main}}$ and ${\omega }_n^{\eta ,\mathrm{main}}$ to the cloud. The cloud receives the local learning model network parameters uploaded by the edge servers and performs weighted aggregation based on the differential contribution evaluation mechanism, then distributes the updated global network parameters to all edge servers.

The cloud determines the contribution of edge server e_n in the parameter aggregation process by evaluating the historical reward variation. Specifically, the cloud extracts several test scenarios from the local database and executes the policy on the global network before and after aggregating the parameters of the server, thereby calculating the reward difference Δr_n(t') between the post-aggregation and pre-aggregation cases. Subsequently, the cloud conducts statistical analysis of the reward variations observed during the historical training process, thereby quantifying the effective contribution of the server in long-term training. The historical contribution score is expressed as

$${\tilde{\pi }}_n={\displaystyle \sum _{t^{\prime }=1}^t\frac{e^{\frac{t^{\prime }-1}{t-1}}-1}{e-1}\Delta r_n(t^{\prime })}$$

(34)

The historical contribution score is further normalized to obtain the final score ${\pi }_n={\displaystyle \frac{{\tilde{\pi }}_n}{{\displaystyle \sum _n{\tilde{\pi }}_n}}}$.

The cloud performs differential aggregation of model parameters based on the historical contribution scores of each edge server, which are expressed as

$${\omega }_{\mathrm{Global}}^{Q,\mathrm{main}}={\displaystyle \frac{1}{N}}{\displaystyle \sum _{n=1}^N{\pi }_n{\omega }_n^{Q,\mathrm{main}}}$$

(35)

$${\omega }_{\mathrm{Global}}^{\eta ,\mathrm{main}}={\displaystyle \frac{1}{N}}{\displaystyle \sum _{n=1}^N{\pi }_n{\omega }_n^{\eta ,\mathrm{main}}}$$

(36)

The aggregated global model is transmitted by the cloud to every edge server, where the edge servers overwrite their own network parameters with the aggregated parameters, i.e., ${\omega }_n^{Q,\mathrm{main}}={\omega }_{\mathrm{Global}}^{Q,\mathrm{main}}$, ${\omega }_n^{\eta ,\mathrm{main}}={\omega }_{\mathrm{Global}}^{\eta ,\mathrm{main}}$. Meanwhile, the local target networks undergo soft updates to maintain stable learning, which are formulated as

$${\omega }_n^{Q,\mathrm{target}}\leftarrow \xi {\omega }_n^{Q,\mathrm{main}}+(1-\xi ){\omega }_n^{Q,\mathrm{target}}$$

(37)

$${\omega }_n^{\eta ,\mathrm{target}}\leftarrow \xi {\omega }_n^{\eta ,\mathrm{main}}+(1-\xi ){\omega }_n^{\eta ,\mathrm{target}}$$

(38)

where ξ is the soft update coefficient, which is a positive number much smaller than 1.

For each federated aggregation round, the differential contribution evaluation requires testing the candidate model associated with each participating edge server over J representative scenarios. Therefore, for N participating edge servers, the serial computational complexity is approximately O(NJC_eval), where C_eval denotes the cost of one model evaluation. For fixed J, the evaluation complexity increases linearly with N. For city-wide deployment, the proposed aggregation mechanism can be implemented hierarchically by partitioning the distribution network into multiple regional aggregation domains. Differential contribution evaluation and differentiated aggregation are first performed independently within each region, after which the regional models are further aggregated using the same contribution-evaluation mechanism. This decomposes the city-wide aggregation task into smaller parallel aggregation processes without changing the core principle of the proposed method. In addition, differential evaluation is performed only during periodic federated aggregation rounds rather than in the per-slot real-time scheduling path. During the evaluation process, edge servers continue to make decisions using the latest available model. Therefore, the resulting overhead mainly affects the model-update interval rather than being directly added to the end-to-end service delay. The hierarchical regional implementation further reduces the scale of each evaluation task and supports parallel processing, thereby preserving the latency advantage of continuous edge-side inference.

4.2. Lagrangian Algorithm-Based Cloud–Edge Computing Resource Allocation Optimization Method

The subproblems of edge and cloud computing resource allocation are formulated separately, expressed as

$$\begin{array}{l}\mathrm{SP}2\text{-}1:\underset{\left\{f_{m,n}^{\mathrm{edg}}\left(t\right)\right\}}{\min }{\psi }_n^{\mathrm{edg}}(t)={\displaystyle \sum _{m=1}^{M_n}{S}_{m,n}^{\mathrm{edg}}(t)}\hfill \\ \mathrm{s}.\mathrm{t}.C_4:{\displaystyle \sum _{m=1}^{M_n}{f}_{m,n}^{\mathrm{edg}}\left(t\right)}\le f_{n,\max }^{\mathrm{edg}}\left(t\right),\forall n,\forall t\hfill \end{array}$$

(39)

$$\begin{array}{l}\mathrm{SP}2\text{-}2:\underset{\left\{f_{m,n}^{\mathrm{clo}}\left(t\right)\right\}}{\min }{\psi }^{\mathrm{clo}}(t)={\displaystyle \sum _{n=1}^N{\displaystyle \sum _{m=1}^{M_n}{S}_{m,n}^{\mathrm{clo}}(t)}}\hfill \\ \mathrm{s}.\mathrm{t}.C_5:{\displaystyle \sum _{n=1}^N{\displaystyle \sum _{m=1}^{M_n}{f}_{m,n}^{\mathrm{clo}}\left(t\right)}\le f_{\max }^{\mathrm{clo}}\left(t\right),\forall d_{m,n},\forall t}\hfill \end{array}$$

(40)

For the two computing-resource allocation subproblems in (39) and (40), the Lagrangian method is adopted to transform the constrained resource-allocation problem through dual decomposition, enabling the corresponding variables and multipliers to be updated iteratively. Under the splitting-ratio and computing-paradigm decisions determined by SP1, the KKT conditions are used to characterize the solution of the continuous resource-allocation subproblem. It should be emphasized that these optimality conditions apply only to the resource-allocation subproblem under the given SP1 decisions and do not establish global optimality of the original mixed-integer problem P2.

4.3. Computational Complexity Analysis

Let $𝒫$_η and $𝒫$_Q denote the total numbers of trainable parameters in the actor and critic networks, respectively, which are determined by the input/output dimensions and the number and width of all hidden layers. For the mini-batch size K defined in Equation (27), one actor–critic parameter update has a computational complexity of O[K($𝒫$_η + $𝒫$_Q)]. For N_ep training episodes and T slots per episode, the FDDPG training complexity at each edge server is O[N_epTK($𝒫$_η + $𝒫$_Q)]. The additional complexity of experience sampling and target-network soft updating is of a lower order. For SP2, assuming I_Lag Lagrangian dual iterations, the resource-allocation complexity is $O[I_{\mathrm{Lag}}{\displaystyle \sum _{n=1}^N{M}_n}]$. Hence, after model training, the online computation mainly consists of actor-network inference and the iterative SP2 solution.

5. Simulation

This paper employs the Matlab R2025a simulation tool to simulate the cloud–edge collaborative dynamic resource allocation scenario for efficient distribution automation. The terminal layer comprises 60 DTUs installed across distribution-network equipment, including switch stations and ring main units. The edge layer contains 10 edge servers, while terminal data are delivered to these servers through 5G links. Rayleigh fading is utilized to model the time-varying channel gain fluctuations caused by multipath propagation in wireless communications for distribution networks. Each 5G channel provides a bandwidth of 10 MHz, and the transmission power varies within the range of [40, 70] mW. The data arrival volume A_m,n(t) of each DTU follows a Poisson distribution with a mean of 5 kB per 100 ms slot. The detailed simulation parameters are listed in Table 2 [33,34,35].

Table 2. Simulation parameters.

Parameters	Value	Parameters	Value
T	200	τ	100 ms
M	60	N	10
${\mu }^Q$	0.001	${\mu }^{\eta }$	0.001
ϖE	0.3	ϖD	0.5
$f^{\mathrm{d}}\left(t\right)$	0.5 GHz	ϖC	0.2
$S_{\max }^{\mathrm{edg}}$	300 ms	$f_{n,\max }^{\mathrm{edg}}\left(t\right)$	10 GHz
$S_{\max }^{\mathrm{d}}$	200 ms	$f_{\max }^{\mathrm{clo}}\left(t\right)$	60 GHz
$S_{\max }^{\mathrm{clo}}$	300 ms	Slots per episode T	200
Actor network	256–128 fully connected	Critic network	256–128 fully connected
Hidden activation	ReLU	Random seeds	1, 7, 21, 42, 2025
Optimizer	Adam	Discount factor γ	0.95
Soft-update coefficient ρ	0.01	Mini-batch size K	64
Success pool capacity	1 × 104	Failure pool capacity	1 × 104
Noise scale χ	0.1	Noise sensitivity κ	0.05
Training episodes E	500	Slots per episode T	200

Two baseline algorithms are selected as benchmarks. Baseline 1 adopts a federated reinforcement learning framework for cloud–edge–end collaborative optimization. In this approach, federated DRL runs on edge servers to jointly determine task offloading and computing resource allocation, while the cloud periodically combines the global model [33]. Baseline 2 employs a DDPG-driven cloud–edge collaborative optimization framework with centralized training and distributed execution. In this approach, DDPG models are distributed across edge servers to dynamically optimize computing resource allocation and task offloading [34].

Figure 3 illustrates how the end-to-end delay evolves across successive time slots. The results indicate that all three algorithms exhibit a downward trend and eventually converge. Relative to Baseline 1 and Baseline 2, the proposed algorithm lowers the average end-to-end delay by 2.01% and 0.08%, respectively. This is because the proposed algorithm adopts a 3D backpressure-aware FDDPG approach, which enhances the policy’s ability to perceive the balance of queue pressure across multiple layers. Consequently, it dynamically adjusts the data splitting ratio and computation paradigm, thereby providing low-delay resource scheduling support in distribution automation scenarios.

Figure 3. End-to-end delay of terminals versus slots under different algorithms.

Figure 4 shows the variation in the normalized reward with training episodes under different algorithms. As shown in the results, the proposed algorithm converges to a steady-state normalized reward of 0.9210, while Baseline 1 and Baseline 2 converge to 0.5851 and 0.5304, improving the reward by 57.40% and 73.65%, respectively. This improvement stems from the joint effects of the proposed 3D backpressure-aware mechanism, dual experience replay mechanism, and cloud-side federated aggregation method based on differentiated contribution evaluation, which evaluates the models of each edge server, quantifies their parameter contributions during aggregation, and increases the parameter weights of high-performance edge servers to achieve differentiated aggregation. This approach also reduces the adverse influence of low-quality or abnormal parameters on the global model, speeds up federated aggregation convergence, improves the global model’s overall performance, and maintains the stability of federated learning.

Figure 4. Normalized reward of different algorithms versus slots.

Figure 5 shows the virtual queue deficit under different algorithms. It can be observed from the results that, compared with baseline 1 and baseline 2, the proposed algorithm maintains a relatively low level of average virtual queue deficit at the terminal, edge, and cloud layers, with overall smaller fluctuations. This is because the proposed algorithm leverages a 3D backpressure-aware mechanism to capture the queue differences across the terminal–edge–cloud hierarchy in real time and introduces the “local–edge” and “local–cloud” dual backlog gaps as dynamic weighting factors in the reward function. In this way, the agent can instantly perceive the congestion degree at each layer during every decision slot and adaptively adjust the data splitting ratio and computation paradigm selection, thereby effectively suppressing queue expansion and significantly enhancing system stability and robustness under high-concurrency scenarios.

Figure 5. Virtual queue deficit under different algorithms.

Figure 6 illustrates the decomposition of the end-to-end delay of the three algorithms under different numbers of terminals. According to (14), the end-to-end delay is decomposed into the local queuing delay and the dominant offloaded-path delay, i.e., $S_{m,n}^d(t)+\max \{S_{m,n}^{\mathit{edg}}(t),S_{m,n}^{\mathit{clo}}(t)\}$. The results reveal that with only a small number of terminals, the three algorithms produce nearly identical delay performance. However, as the number of terminals increases, the end-to-end delay of all algorithms rises, though at significantly different rates and with varying allocation characteristics. At 80 terminals, the proposed algorithm lowers the total delay by approximately 7.04% relative to Baseline 1 and by 8.63% relative to Baseline 2. This improvement mainly arises from its capacity to distribute the task load more evenly among the terminal, edge, and cloud layers according to their individual computing capabilities. In contrast, Baseline 1 and Baseline 2 show clear imbalances in allocation, leading to bottlenecks in certain layers and increasing the overall delay. The proposed algorithm, however, employs a 3D backpressure mechanism to identify queue imbalances in real time and adaptively modify task offloading and resource allocation. As a result, tasks can be assigned more flexibly, and the global delay is effectively decreased.

Figure 6. Decomposition of the end-to-end delay under different numbers of terminals.

To further evaluate the proposed algorithm under realistic distribution automation conditions, four groups of robustness experiments are conducted, as shown in Figure 7. Each setting is independently repeated ten times with different random seeds, and the mean end-to-end delay is reported. The nominal operating point is kept consistent with Figure 3, in which the proposed algorithm achieves an average end-to-end delay of 485.0 ms and lowers the delay by 6.51% and 8.82% compared with Baseline 1 and Baseline 2, respectively.

Figure 7. Robustness of different algorithms under dynamic and heterogeneous operating conditions.

Figure 7a evaluates the influence of wireless uncertainty and communication failures. Rayleigh block fading is generated across time slots, while the additional penetration loss increases from 0 to 30 dB and the random link interruption probability increases from 0 to 8% from W0 to W3. The results indicate that the end-to-end delay of all three algorithms rises as the wireless environment deteriorates. Under W3, the proposed algorithm achieves 548.5 ms, which is 9.11% and 12.52% lower than Baseline 1 and Baseline 2, respectively. This is because the 3D backpressure-aware mechanism perceives in real time the queue accumulation caused by transmission degradation and retransmission, and accordingly adjusts the terminal splitting ratio and computing paradigm to redistribute the processing load across the terminal, edge, and cloud layers. The proposed FDDPG policy is not tied to a specific fading distribution. Severe attenuation and multipath interference can be represented through the corresponding instantaneous channel gain and effective transmission rate in the communication model. Their effects are subsequently propagated to the transmission delay, queue evolution, and virtual-queue states, which are observed by the learning model and reflected in the reward. Therefore, alternative channel models can be incorporated without changing the FDDPG architecture or decision variables, providing compatibility with different wireless propagation environments.

Figure 7b investigates the effect of heterogeneous DTU computing capabilities. The mean terminal computing capability is kept at 0.5 GHz, while its dispersion is gradually enlarged from a fixed value to U[0.2,0.8] GHz. The delay of all algorithms increases with terminal heterogeneity, since low-capability DTUs tend to become local processing bottlenecks. Nevertheless, the proposed algorithm maintains the lowest delay and reaches 519.0 ms under the largest dispersion, which is 8.63% and 11.73% lower than the two baselines. The reason is that the terminal splitting ratio is optimized for each DTU rather than uniformly, so that a smaller local share is assigned to low-capability terminals, thereby suppressing the straggler effect.

Figure 7c evaluates different traffic loads by varying the mean data arrival volume of DTUs from 3 to 9 kB per 100 ms slot. With light traffic, the three algorithms exhibit similar delay performance. As the traffic load increases, queue accumulation becomes significant and the performance gap widens. At 9 kB per slot, the proposed algorithm achieves 694.0 ms, whereas Baseline 1 and Baseline 2 reach 768.0 ms and 794.0 ms, corresponding to reductions of 9.64% and 12.59%, respectively. This demonstrates that the 3D backpressure mechanism provides greater benefit under high-load conditions.

Figure 7d further examines the influence of the model synchronization delay, which is varied from 0 to 200 ms. For the proposed algorithm and Baseline 1, this delay corresponds to the federated aggregation and model distribution process, whereas for Baseline 2 it corresponds to the parameter synchronization between the centralized training server and the distributed execution agents. During a delayed synchronization round, each edge server continues to execute the latest available local policy until the updated model is received. Although model staleness gradually increases the end-to-end delay, the proposed method remains comparatively insensitive to this delay. At a synchronization delay of 200 ms, its average end-to-end delay is 507.5 ms, which remains 8.39% and 11.20% lower than Baseline 1 and Baseline 2, respectively.

To quantify the contribution of each proposed component, an ablation study is conducted, as shown in Figure 8, where the steady-state level of each curve reflects the delay performance and the training episode at which the curve becomes flat reflects the convergence speed. Each variant is independently repeated ten times with different random seeds, and the mean results are reported. Removing the 3D backpressure mechanism degrades the end-to-end delay performance by approximately 4.98%, confirming its contribution to multi-layer congestion-aware load balancing. Removing the dual experience replay mechanism degrades the delay performance by 4.02% and reduces the convergence efficiency by 21.54%, indicating its major role in accelerating policy learning. Removing the differential contribution-based aggregation results in a 1.76% degradation in delay performance and a 10.08% reduction in convergence efficiency, demonstrating its contribution to federated-training consistency and stability. Overall, the complete proposed method achieves an 8.80% delay improvement over standard FDDPG, verifying the complementary effects of the three mechanisms.

Figure 8. Convergence of the average end-to-end delay under different ablation variants.

To investigate the influence of the 3D backpressure weighting factor, ϑ is varied from 0 to 1.5 while all the other parameters remain unchanged. Each setting is independently repeated ten times with different random seeds, and the mean end-to-end delay is reported. In addition, ${\Gamma }_{\vartheta }={\displaystyle \sum _t|}\vartheta {\Phi }_n(t)|/{\displaystyle \sum _t|}{\psi }_n(t)|$ is introduced to quantify the relative contribution of the backpressure shaping term to the original SP1 objective. As shown in Figure 9, increasing ϑ from zero initially improves the delay performance by strengthening the explicit inter-layer pressure guidance, and the case of ϑ = 0 is consistent with the variant without the 3D backpressure mechanism in Figure 8. The performance remains relatively stable over the moderate range of 0.15–0.60, within which the maximum deviation from the best result is only 0.85%. When ϑ becomes large, Γ_ϑ approaches or exceeds unity and the delay increases by 3.88% at ϑ = 1.5, indicating that excessive emphasis on instantaneous pressure differences may distort the original delay-and-stability objective. These results provide a practical parameter-setting guideline, i.e., the backpressure term should serve as an auxiliary directional signal with a moderate contribution relative to ψ_n(t), rather than becoming the dominant component of the reward. Accordingly, ϑ = 0.30 is adopted in this paper.

Figure 9. Sensitivity of the end-to-end delay and the relative backpressure contribution Γ_ϑ to the weighting factor ϑ.

6. Conclusions

In this paper, we develop a 3D backpressure learning-driven cloud–edge collaborative framework for dynamic resource allocation, targeting limited computing capacity, communication bottlenecks, and time-varying wireless channels in distribution automation. The proposed method optimizes terminal splitting, computing paradigm selection, and resource allocation using Lyapunov optimization and a two-stage solution framework. Simulation results show that the proposed method achieves lower end-to-end delay than Baseline 1 and Baseline 2, with reductions of 6.51% and 8.82%, respectively. Furthermore, the dual experience replay mechanism accelerates convergence by 21.54% while decreasing end-to-end delay by 4.02%. These improvements demonstrate the effectiveness and reliability of the proposed approach in resource scheduling. The current framework mainly focuses on the performance of cloud–edge collaborative resource allocation, while cybersecurity protection for model and information exchange should be further strengthened to enhance the security and reliability of cloud–edge collaboration. Future work will investigate secure aggregation, malicious-update detection, and trustworthy communication mechanisms to further improve the cybersecurity and practical reliability of the proposed framework.