Benchmarking MARL for UAV-Assisted Mobile Edge Computing Under Realistic 3D Collision Avoidance Navigation Constraints for Periodic Task Offloading

Abstract

The rapid growth of Internet of Things (IoT) and Industrial IoT applications has intensified the demand for low-latency and reliable computation support for deadline-constrained periodic real-time tasks. While unmanned aerial vehicles (UAVs) enabling mobile edge computing (MEC) can reduce latency by bringing compute closer to data sources, terrestrial MEC deployments often suffer from limited coverage and poor adaptability to spatially heterogeneous demand. In this paper, we study a multiple-UAV-assisted MEC system serving cluster-based IoT networks, where cluster heads generate deadline-constrained periodic tasks for offloading under strict deadlines. To ensure practical feasibility in dense urban environments, we benchmark UAV mobility using a realistic 3D collision avoidance navigation graph with shortest-path execution, rather than assuming unconstrained continuous UAV motion in free space. On top of this benchmark, we systematically compare three multi-agent reinforcement learning (MARL) paradigms for joint navigation and periodic task offloading: (i) continuous 3D control MARL that outputs motion commands directly; (ii) discrete graph-based MARL that selects collision-free shortest paths; and (iii) asynchronous macro-action MARL. Using a high-fidelity 3D digital twin of San Francisco, we evaluate these paradigms under a unified protocol in terms of offloading success, end-to-end latency, and energy consumption. The results reveal clear performance trade-offs induced by realistic 3D collision avoidance constraints and provide actionable insights for designing UAV-assisted MEC systems supporting periodic real-time task offloading.

1. Introduction

The rapid proliferation of Internet of Things (IoT) has led to a substantial increase in computation-intensive and delay-sensitive workloads generated by mobile devices (IMDs). Many emerging services, including real-time perception, mobile augmented reality, industrial inspection, and safety monitoring, impose stringent end-to-end latency requirements and demand reliable computation support. While mobile edge computing (MEC) can potentially reduces latency by deploying computation capability near data sources [1,2,3,4], existing terrestrial MEC systems are often sparsely deployed, capacity-limited, and unable to dynamically adapt to spatially heterogeneous network loads. These limitations are particularly evident in large industrial campuses, smart factories, and complex outdoor environments, where IMDs are densely distributed but geographically dispersed.

Unmanned aerial vehicles (UAVs) equipped with lightweight edge servers have emerged as a flexible complement to fixed MEC infrastructure, offering on-demand deployment flexibility, controllable altitude, and favorable line-of-sight (LoS) communication [5,6]. Their mobility enables them to provide computation and communication services in remote, infrastructure-deficient, or temporary hotspot regions where conventional MEC fails to meet the reliability or latency requirements. Existing studies have shown that UAVs can effectively enhance MEC performance by improving communication rates, expanding service coverage, and reducing network energy consumption [7,8]. Recent studies have further investigated large-scale multi-UAV-enabled MEC systems and layered UAV-assisted cloud-edge architectures, showing that joint optimization of association, deployment, trajectory, caching, and offloading can substantially improve energy efficiency and latency performance [9,10,11,12,13].

Meanwhile, cluster-based IoT architectures have been widely adopted to reduce signaling overhead and enhance scalability [14,15]. In such architectures, IMDs within the same region forward sensing or computation tasks to a cluster head (CH), and only CHs interact with MEC servers. This hierarchical structure aligns naturally with UAV-assisted MEC systems, as UAVs can efficiently serve CHs rather than all ground devices. Moreover, many MEC studies explicitly model real-time periodic workloads, where mobile devices execute periodic tasks (possibly mixed with aperiodic or sporadic ones) and make deadline-aware offloading decisions over hyperperiods [16]. Recent work has further considered 6G-empowered MEC architectures that offload periodic and sporadic real-time tasks via RL-based policies across multi-tier edge–cloud resources [17]. These studies highlight the importance of explicitly capturing periodic task structures and deadlines when designing offloading policies for real-time applications.

In addition to communication and offloading decisions, the physical mobility of UAVs plays a critical role in determining the effectiveness of UAV-assisted IoT/MEC systems. Prior studies have demonstrated that jointly optimizing UAV deployment and flight trajectories can significantly reduce energy consumption and improve service efficiency, for example by introducing foothold-based deployment strategies and precomputed trajectory planning mechanisms [9,18]. However, most existing work mainly focuses on optimization of service locations and path lengths in abstract or simplified spaces, without explicitly addressing the safety and feasibility challenges caused by complex 3D urban structures. Recent studies on urban drone freight delivery further suggest that UAVs are becoming a promising solution for time-sensitive logistics in congested city environments while also highlighting the importance of safe 3D route organization and scalable aerial coordination in complex urban airspaces [19,20].

In realistic urban and industrial environments, directly controlling UAVs in continuous 3D space can easily produce unsafe or infeasible trajectories, most notably building collisions, which may cause task offloading failures and even safety hazards. To ensure practical route feasibility while still exploiting the mobility benefits of UAVs, we represent the service space as a 3D navigation graph constructed from no-collision regions and precomputed shortest paths. Building on this benchmark, we conduct a systematic comparison of three representative solution paradigms and their typical learning instantiations, aiming to characterize the conditions under which each paradigm is effective or ineffective.

The main contributions of this paper are summarized as follows:

• We formulate a multi-UAV-assisted MEC problem for periodic task offloading in a realistic 3D city digital twin of San Francisco with explicit collision avoidance constraints, where cluster heads generate deadline-constrained periodic tasks and UAVs provide mobile edge service. The benchmark models feasible 3D navigation and routing via a collision-free navigation graph with shortest-path execution, enabling quantitative evaluation of offloading performance under stringent timing and energy constraints.
• We identify and evaluate three MARL paradigms for solving the above problem: (i) continuous 3D control MARL that directly outputs motion commands; (ii) discrete decision-making MARL over a predefined collision-free navigation graph with shortest-path execution; and (iii) asynchronous macro-action MARL where each agent selects a new action only upon macro-action completion, leading to naturally de-synchronized decision epochs. For each paradigm, we implement representative algorithms and design a unified evaluation protocol, revealing key performance trends in offloading success, latency, energy consumption, and overall efficiency.
• We release reproducible resources, including a 3D digital-twin environment, collision-free navigation maps, and periodic task offloading generators, to facilitate future research on UAV–MEC systems under realistic 3D collision avoidance constraints.

The remainder of this paper is organized as follows. Section 2 reviews related work. Section 3 introduces the system architecture and models. Section 4 formulates the benchmark problem and describes the three representative solution paradigms. Section 5 presents the evaluation protocol and discusses the experimental results. Section 6 provides further insights, limitations, and future directions. Section 7 concludes the paper. For easy reference, notations mainly used in this article are listed in Table 1.

Table 1. List of important notations used in this paper.

Notation	Description
D	3D navigation domain
$k,K,\mathcal{K}$	Index; total number; set of IMDs
$u,U,\mathcal{U}$	Index; total number; set of UAVs
T	Duration of each period
$I_k$	Computation task of IMD k
$L_k$	Input data size of task $I_k$
$O_k$	Number of output data per input bit of task $I_k$ ($O_k<1$)
$C_k$	CPU cycles required for computing one bit of task $I_k$
$t_k^{\max }$	Maximum permissible latency for task $I_k$
${\mathbf{q}}_u\left(t\right)$	Trajectory of UAV u
${\mathbf{w}}_k$	Position of IMD k
$v_{\max }$	UAV’s maximum flight speed
$h_{u,k}\left(t\right)$	Average channel power gain between UAV u and IMD k at time t
B	Communication bandwidth
${\sigma }^2$	Noise power at receivers
$f_k^{\mathrm{loc}}$	Computing capacity of IMD k
$f^c$	Computing capacity of the UAV’s edge server
${\gamma }_k^{\mathrm{loc}}$	Coefficient related to IMD k’s processor
${\gamma }_c$	Coefficient related to edge server’s processor
$P_{k,\mathrm{ul}}$	IMD’s uplink transmit power
$P_{u,\mathrm{dl}}$	UAV’s downlink transmit power
$P_{k,\mathrm{dl}}$	IMD’s downlink receive-circuit power
$P_{u,\mathrm{ul}}$	UAV’s uplink receive-circuit power

2. Related Work

2.1. UAV-Assisted MEC

Existing studies have investigated UAV-assisted MEC from several representative perspectives. For instance, energy-efficient trajectory planning and computation offloading have been studied in UAV-enabled IoT networks, where transformer-based models or joint bit–energy optimization were used to reduce latency and energy consumption [21,22]. Other works investigate UAV-assisted task offloading in mobile or vehicular edge systems, in which UAVs dynamically assist overloaded terrestrial infrastructures via online optimization under long-term energy constraints [23]. Furthermore, multi-UAV cooperative offloading and resource allocation have been examined in next-generation aerial computing systems, focusing on cooperative scheduling, partial offloading, and load balancing among multiple UAVs [24].

Building on these directions, subsequent studies have moved toward more comprehensive joint optimization settings. These include security-aware co-design of multi-UAV deployment, task offloading, and service placement under eavesdropping or jamming threats [25,26], collaborative multi-UAV offloading and resource allocation with DRL-based coordination [10,11,12], and joint deployment, scheduling, and path planning designs in complex environments such as urban disaster scenarios and high-density mobile device settings [27,28,29]. Overall, these studies show that UAV-assisted MEC is evolving from simplified offloading models toward integrated designs that jointly consider deployment, trajectory planning, communication, security, and resource scheduling in increasingly dynamic environments.

2.2. Multi-Agent Reinforcement Learning

Multi-agent reinforcement learning (MARL) has been widely adopted for decentralized coordination problems [30,31,32,33,34,35], where centralized training with decentralized execution (CTDE) is commonly used to stabilize learning. However, most commonly used MARL benchmarks and algorithm implementations are built around synchronized primitive-action steps, which limits their applicability in domains with heterogeneous action durations and asynchronous agent behaviors. To address this issue, macro-action-based MARL frameworks have been proposed, enabling agents to select temporally extended actions composed of multiple low-level steps. In particular, Xiao et al. [36] introduced a decentralized macro-action-based deep Q-learning approach with centralized training, demonstrating improved coordination efficiency. Subsequently, asynchronous actor–critic methods were developed to explicitly handle de-synchronized decision epochs and varying macro-action completion times [37]. More recently, the Agent-Centric Actor–Critic (ACAC) framework further improved learning stability in asynchronous MARL by avoiding information padding through agent-centric history encoding [38].

Beyond general MARL, recent UAV-assisted MEC studies have increasingly adopted multi-agent DRL to jointly optimize mobility, offloading, communication, and resource control. Representative examples include MATD3- and MADDPG-based frameworks for secure offloading, deployment, and resource allocation [10,25,26], as well as decentralized MADRL with learned inter-UAV message passing for scalable coordination under partial observability [12]. These studies suggest that MARL is a promising tool for multi-UAV MEC control, but most existing studies still emphasize task-specific joint optimization rather than systematically comparing different decision paradigms under realistic 3D motion feasibility constraints. This gap motivates our comparison of continuous control, graph-based discrete decision-making, and asynchronous macro-action MARL within a unified benchmark.

3. System Model

As illustrated in Figure 1, we study a multi-UAV enhanced edge network in which each UAV carries an integrated edge server. These UAVs collaboratively provide edge services to ground IMDs and traverse distinct spatial regions to serve geographically distributed devices. Since each UAV must both move within the scene and support task uploading and result downloading, the system must carefully balance the energy consumption between propulsion for aerial mobility and computation/communication for service delivery.

Figure 1. Multi-UAV enhanced edge network.

To mitigate interference during concurrent task offloading between multiple IMDs and UAVs as well as to reduce communication overhead, we group IMDs into multiple clusters. Within each cluster, nodes organize into a forwarding tree based on compressive data gathering, which allows each IMD to transmit its task data to a designated cluster head (CH). Communication costs between individual IMDs and their CHs are assumed negligible. Each UAV directly engages in communication and offloading only with the CH of each cluster.

Within each cluster, the cluster head (CH) aggregates the task streams generated by its member IMDs. To model the CH’s workload, we focus on real-time periodic tasks with quasi-periodic activations, as illustrated in Figure 2, capturing the regular sensing, monitoring, and reporting behaviors commonly observed in IoT systems. Following established models [39,40], we assume homogeneous task periods across IMDs within the same cluster, providing a realistic yet analytically tractable approximation of typical computation workloads generated by IMDs. Unless otherwise stated, these periodic tasks are executed locally by default; when a serving UAV is available, the tasks may be offloaded and executed on the UAV. Local execution alone may result in missed deadlines, motivating opportunistic offloading. Communications among IMDs within the same cluster are considered negligible, and each UAV communicates solely with CHs.

Figure 2. Timeline of a periodic task, showing local computation and the optional edge-compute interval within a task period.

3.1. Communication Model

We consider a group of UAVs $\mathcal{U}\triangleq \{u\mid u=1,2,\dots ,U\}$ that navigate within a prescribed 3D feasible navigation domain $D\subset {\mathbb{R}}^3$. Over each scheduling period of duration T, UAVs maintain connectivity to devices within its coverage. Each IoT mobile device $k\in \mathcal{K}$ is associated with a computation-intensive, delay-sensitive task $I_k=\{L_k,C_k,t_k^{\max }\}$, where $L_k$ (bits) is the input data size, $C_k$ (CPU cycles/bit) is the per-bit computational demand, and $t_k^{\max }$ is the maximum tolerable end-to-end latency. All tasks must be completed within a single period T.

We adopt a binary offloading model: every task is executed either entirely locally on the IMD or entirely offloaded to the UAV. We denote by ${\mathcal{J}}_0$ the set of locally executed tasks and by ${\mathcal{J}}_1$ the set of offloaded tasks, with ${\mathcal{J}}_0\cap {\mathcal{J}}_1=⌀$ and ${\mathcal{J}}_0\cup {\mathcal{J}}_1=\mathcal{K}$. Since modeling data transmission under dynamically varying channel conditions induced by UAV motion is beyond the scope of this work, we assume that both uplink and downlink transmissions occur only during hovering. For an offloaded task $I_k$, uplink transmission delivers all $L_k$ input bits to the UAV during hovering. Edge execution commences immediately after the full input is received, and the resulting output—of $O_k{L}_k$ bits with output-to-input ratio $O_k<1$—is then returned to IMD k via the downlink.

The Euclidean distance between UAV u and IMD k at time t is defined as:

$$d_{u,k}\left(t\right)={∥{\mathbf{q}}_u\left(t\right)-{\mathbf{w}}_k∥}_2.$$

(1)

In air-to-ground communication, the line-of-sight (LoS) propagation typically dominates. For analytical tractability, we ignore large-scale shadowing and small-scale fading, and adopt a free-space path loss model with path-loss exponent $\alpha =2$. Under this model, the average channel power gain between UAV u and IMD k at time t is:

$$h_{u,k}\left(t\right)={\displaystyle \frac{h_0}{{∥{\mathbf{q}}_u\left(t\right)-{\mathbf{w}}_k∥}_2^2}},$$

(2)

where $h_0$ denotes the channel power gain at a reference distance $1 \mathrm{m}$.

3.2. Task Offloading Model

3.2.1. Local Execution

If cluster head (CH) IMD k decides to execute its task $I_k$ locally (i.e., $k\in {\mathcal{J}}_0$), the execution time is

$$t_k^{\mathrm{loc}}={\displaystyle \frac{C_k{L}_k}{f_k^{\mathrm{loc}}}},$$

(3)

where $f_k^{\mathrm{loc}}$ denotes the local CPU frequency of IMD k. Assuming dynamic power dominates the processor energy, the local execution energy is

$$E_k^{\mathrm{loc}}={\gamma }_k^{\mathrm{loc}}{C}_k{L}_k{\left(f_k^{\mathrm{loc}}\right)}^2,$$

(4)

where ${\gamma }_k^{\mathrm{loc}}$ is a chip-dependent coefficient of IMD k.

3.2.2. Edge Execution

For each CH IMD $k\in {\mathcal{J}}_1$ that offloads its task $I_k$ to the UAV for edge computing, the task $I_k$ is defined as the aggregation of all individual tasks generated by IMDs in cluster k during the current period. That is, the CH IMD is responsible for collecting, aggregating, and offloading the composite task $I_k$ to the UAV.

If CH IMD k offloads its task $I_k$ to the UAV for edge computing (i.e., $k\in {\mathcal{J}}_1$), we let $f^c$ and ${\gamma }^c$ denote the CPU frequency and chip coefficient of the UAV’s edge server, respectively. Here, $f_k^{\mathrm{loc}}$ and $f^c$ are abstract computing-capacity parameters rather than exact hardware specifications. As in prior UAV-assisted MEC studies [23,25], the UAV is modeled as a stronger edge server than individual ground devices. The corresponding edge computing time and energy are

$$t_k^{\mathrm{ec}}={\displaystyle \frac{C_k{L}_k}{f^c}},$$

(5)

$$E_k^{\mathrm{ec}}={\gamma }^c{C}_k{L}_k{\left(f^c\right)}^2.$$

(6)

For communication, denote the IMD’s uplink (UL) transmit power by $P_{k,\mathrm{ul}}$, the UAV’s downlink (DL) transmit power by $P_{u,\mathrm{dl}}$, the UAV’s UL receive-circuit power by $P_{u,\mathrm{ul}}$, and the IMD’s DL receive-circuit power by $P_{k,\mathrm{dl}}$. Assuming Gaussian signaling, the instantaneous spectral efficiencies (in bps/Hz) of the uplink and downlink channels at time t are expressed as

$$R_k^{\mathrm{ul}}\left(t\right)={\log }_2\left(1+{\displaystyle \frac{P_{k,\mathrm{ul}}{h}_{u,k}\left(t\right)}{{\sigma }^2}}\right),$$

(7)

$$R_k^{\mathrm{dl}}\left(t\right)={\log }_2\left(1+{\displaystyle \frac{P_{u,\mathrm{dl}}{h}_{u,k}\left(t\right)}{{\sigma }^2}}\right),$$

(8)

where ${\sigma }^2$ denotes the receiver noise power.

Note that both uplink and downlink transmissions are executed only when the UAV hovers. During this period, the wireless channel is assumed to remain quasi-static, and hence the total transmission times for the uplink and downlink are

$$t_k^{\mathrm{ul}}={\displaystyle \frac{L_k}{B{\overline{R}}_k^{\mathrm{ul}}}},t_k^{\mathrm{dl}}={\displaystyle \frac{O_k{L}_k}{B{\overline{R}}_k^{\mathrm{dl}}}},$$

(9)

where B is the channel bandwidth.

The corresponding energy consumptions are

$$E_{u,\mathrm{dl}}=P_{u,\mathrm{dl}}{t}_k^{\mathrm{dl}},E_{u,\mathrm{ul}}=P_{u,\mathrm{ul}}{t}_k^{\mathrm{ul}},$$

(10)

The total latency for completing the offloaded task $I_k$ consists of the uplink transmission, edge computation, and downlink transmission times:

$$t_k^{\mathrm{mec}}=t_k^{\mathrm{ul}}+t_k^{\mathrm{ec}}+t_k^{\mathrm{dl}}.$$

(11)

Finally, the total energy consumption at UAV u due to task $I_k$ is:

$$E_u\left(I_k\right)=E_{u,\mathrm{ul}}+E_{u,\mathrm{dl}}+E_k^{\mathrm{ec}}.$$

(12)

3.3. UAV’s Mobility Model

Without loss of generality, the flight trajectory of UAV u over the horizon T is denoted by ${\mathbf{q}}_u\left(t\right)\in D\subset {\mathbb{R}}^3$. Operationally, the trajectory is generated by concatenating a finite sequence of decision-induced subpaths. We let ${\mathrm{\Pi }}_u\triangleq \{{\mathbf{c}}_{u,0},{\mathbf{c}}_{u,1},\dots ,{\mathbf{c}}_{u,S_u}\}$ be the ordered set of waypoints for UAV u, where ${\mathbf{c}}_{u,s}\in D$ typically coincides with a cluster head (CH) position to be visited. As illustrated in Figure 3a, the planned trajectory is computed within the UAV feasible flight domain and is composed of consecutive subpaths connecting cluster heads, reflecting the decision-induced flight planning process. In contrast, Figure 3b shows the realistic 3D collision regions in the urban environment. Note that a single decision step may involve multiple waypoints to be traversed in sequence.

Figure 3. Illustration of UAV mobility and spatial constraints. (a) Planned UAV trajectory in the feasible flight domain. (b) Realistic 3D collision regions in the urban environment.

The sth subpath is the directed segment

$${\mathcal{P}}_{u,s}:{\mathbf{c}}_{u,s}\to {\mathbf{c}}_{u,s+1},s=0,1,\dots ,S_u-1,$$

(13)

and the full trajectory satisfies

$${\mathbf{q}}_u\left(t\right)=\underset{s=0}{\overset{S_u-1}{⨁}}{\mathcal{P}}_{u,s},$$

(14)

where ⊕ denotes concatenation in time. For the straight subpath ${\mathcal{P}}_{u,s}:{\mathbf{c}}_{u,s}\to {\mathbf{c}}_{u,s+1}$, we let $d_{u,s}={\parallel {\mathbf{c}}_{u,s+1}-{\mathbf{c}}_{u,s}\parallel }_2$. Assuming piecewise-constant speed $v_u\left[s\right]$ with $0<v_u\left[s\right]\le v_{\max }$, the per-segment flight time is

$$T_{u,\mathrm{fly}}\left(s\right)={\displaystyle \frac{d_{u,s}}{v_u\left[s\right]}}.$$

(15)

Following [41], the propulsion power for horizontal motion at speed v is

$$\begin{array}{cc}\hfill P_{\mathrm{mov}}\left(v\right)& =P_0\left(1+{\displaystyle \frac{3v^2}{U_{\mathrm{tip}}^2}}\right)+P_1\left(\sqrt{1+{\displaystyle \frac{v^4}{4v_0^4}}}-{\displaystyle \frac{v^2}{2v_0^2}}\right)\hfill \\ \hfill & +{\displaystyle \frac{1}{2}}{d}_0\rho s_0\delta v^3,\hfill \end{array}$$

(16)

where $P_0$ and $P_1$ are the blade-profile and induced powers in hovering, $U_{\mathrm{tip}}$ is the rotor tip speed, $v_0$ is the mean rotor induced velocity in hovering, $d_0$ is the fuselage drag ratio, $s_0$ is the rotor solidity, $\rho$ is the air density, and $\delta$ is the rotor disk area. The per-segment flight energy is

$$E_{u,\mathrm{fly}}\left(s\right)=P_{\mathrm{mov}}\left(v_u\left[s\right]\right)T_{u,\mathrm{fly}}\left(s\right)=P_{\mathrm{mov}}\left(v_u\left[s\right]\right){\displaystyle \frac{d_{u,s}}{v_u\left[s\right]}}.$$

(17)

At a CH waypoint ${\mathbf{c}}_j^u$, the UAV hovers for dwell time $T_{u,\mathrm{hov}}\left(j\right)$. Substituting $v=0$ into (16) gives the constant hover power

$$P_{\mathrm{hov}}=P_0+P_1,$$

(18)

and the hover propulsion energy is

$$E_{u,\mathrm{hov}}\left(j\right)=P_{\mathrm{hov}}{T}_{u,\mathrm{hov}}\left(j\right).$$

(19)

Aggregating along the full trajectory, the total time and energy consumption in one decision epoch are

$$T_{u,\mathrm{act}}=\sum _{s=0}^{S_u-1}{T}_{u,\mathrm{fly}}\left(s\right)+\sum _{j\in {\mathcal{J}}_u}{T}_{u,\mathrm{hov}}\left(j\right),$$

(20)

$$E_{u,\mathrm{act}}=\sum _{s=0}^{S_u-1}{E}_{u,\mathrm{fly}}\left(s\right)+\sum _{j\in {\mathcal{J}}_u}{E}_{u,\mathrm{hov}}\left(j\right),$$

(21)

where ${\mathcal{J}}_u$ indexes the CH waypoints visited by UAV u.

4. Problem Formulation and Solution Paradigms

4.1. Dec-POMDP Modeling

We formulate the multi-UAV scheduling problem as a decentralized partially observable Markov decision process (Dec-POMDP), denoted by $\mathcal{M}=⟨\mathcal{U},\mathcal{S},\mathcal{O},\mathcal{A},P,R,\gamma ⟩$, where $\mathcal{U}$ denotes the set of UAV agents, $\mathcal{S}$ is the global state space, $\mathcal{O}$ is the joint observation space, $\mathcal{A}$ is the joint action space, P is the state transition kernel, R is the reward function, and $\gamma \in (0,1)$ is the discount factor. Specifically, the global state includes the UAV positions, system time, and task queue states, while each agent makes decisions based on its local observation.

4.2. Synchronous Multi-Agent Continuous Control Paradigm for UAV-Assisted Offloading

In UAV-assisted offloading with constant flight altitude, the system is inherently decentralized and partially observable: each UAV makes decisions based on locally available information, while its outcome is coupled with others through shared tasks, spatial competition, and time-varying service opportunities. Compared with the single-agent MDP abstraction, the key distinction in the Dec-POMDP formulation is the presence of decentralized observations and joint-action-induced non-stationarity, which motivates the centralized training with decentralized execution (CTDE) paradigm. Under CTDE, policies remain decentralized at execution, while training can exploit additional centralized information to improve value estimation and credit assignment. For continuous control, MADDPG [35] is a representative and widely used MARL algorithm, and has been frequently adopted in multi-UAV control and UAV-assisted MEC/offloading studies [25,26]. In addition, MAPPO is a representative on-policy method that combines decentralized actors with a centralized critic for coordinated updates, and has shown strong performance in cooperative multi-agent settings [30]. Accordingly, in the continuous control benchmark considered in this work, we include MADDPG together with MAPPO and IPPO as representative learning baselines. Related studies have also incorporated Lyapunov-based per-slot control to address long-term constraints and stochastic dynamics [23], as well as MARL-based coordination mechanisms in multi-UAV MEC systems [42].

Our 3D simulator follows this paradigm by modeling each UAV as an agent with decentralized observations derived from its own state and local neighborhood information. Each observation contains the UAV’s coordinates, the coordinates of all UAVs, the coordinates and potential rewards of the nearest navigable points, and the potential reward of hovering at the current location. Actions are issued synchronously at fixed time steps and are represented by three continuous variables: azimuth, elevation, and distance ratio, which jointly determine the UAV’s movement direction and displacement within one step. After movement, the UAV can serve offloading tasks within a predefined communication range. The reward is shaped by the number of successfully served tasks, together with a small time penalty, while boundary violation is treated as a terminal failure event with a large negative penalty. Under this design, decentralized policies can be executed from local observations, and CTDE-based algorithms such as MAPPO and MADDPG can be applied during training.

4.3. Synchronous Multi-Agent Discrete Control Paradigm for UAV-Assisted Offloading

In obstacle-aware scenarios, a common modeling convention is to construct a collision-free feasible set (e.g., safe corridors or a waypoint graph over the no-collision region) and discretize mobility into a finite set of motion primitives. Each slot corresponds to a synchronous joint decision in which each UAV selects a discrete action, such as moving to one of several nearby feasible waypoints or hovering, yielding a Dec-POMDP with a discrete joint-action space. This setting retains multi-agent non-stationarity and credit-assignment challenges, and is therefore naturally paired with CTDE-based solvers such as categorical-actor MAPPO. As a representative example, Refs. [43,44] adopt PPO with discrete trajectory commands under a constant-altitude UAV motion model; our work follows the same discrete control convention but further extends the simulator to a 3D environment with height variations and 3D spatial constraints.

Our simulator instantiates this paradigm by representing each UAV as an agent with local observations and a discrete mobility action set that selects one of the nearest feasible waypoint transitions or hovering. All UAVs act synchronously at each step; heterogeneous travel/service durations are reconciled by advancing all agents with a common slot length equal to the maximum individual duration and counting the residual as waiting, which preserves a well-defined joint transition. The per-step reward is driven by successful task service within a fixed hovering window with time-discounting and penalties, enabling decentralized execution and straightforward CTDE training with MAPPO.

4.4. Asynchronous Multi-Agent Discrete Macro-Action Paradigm for UAV-Assisted Offloading

In UAV-assisted offloading, enforcing synchronous joint decisions can incur non-negligible waiting overhead: agents that finish earlier must idle until the slowest agent completes its current travel/service, which may reduce effective utilization and introduce additional variance in the learning signal. A widely used alternative is to model high-level decisions as discrete macro-actions with agent-specific, state-dependent durations, so that each UAV starts and terminates decisions asynchronously. This leads to an asynchronous Dec-POMDP abstraction, where policy updates are triggered at macro-action terminations rather than at globally synchronized slots. Recent asynchronous actor–critic formulations explicitly address this setting by enabling direct optimization of asynchronous policies under decentralized, centralized, or CTDE training paradigms, and report improved scalability and solution quality in realistic multi-agent domains [37,38]. In task offloading systems, asynchrony is also a natural modeling choice when task generation instants and server workloads are heterogeneous across UAVs; representative studies adopt asynchronous decision-making and learning-based control to improve timeliness-related objectives under such dynamics [45].

Our 3D simulator is consistent with the asynchronous discrete macro-action MARL setting by decoupling decision times across UAVs and by assigning temporally extended effects to each discrete decision. Specifically, each UAV issues a new decision only at settlement instants, and the committed decision unfolds over a state-dependent duration that covers the induced mobility and a fixed service window, while the environment advances through event-driven updates of arrivals and task completions. Rewards are accumulated during macro-action execution, including task gains and time penalties, and are released only upon macro-action termination, aligning learning signals with temporally extended credit assignment rather than per-slot micro-transitions. Observations remain decentralized and execution-feasible in 3D by combining absolute coordinates, local neighborhood context, and a compact macro-status descriptor, and by refreshing neighborhood features only at settlement points to match asynchronous decision boundaries. This design avoids enforcing global synchronization at every step, thereby reducing waiting overhead, while preserving well-defined transitions and termination signals suitable for asynchronous actor–critic training in multi-UAV offloading.

5. Simulation and Results Analysis

We consider a digital-twin scenario based on San Francisco, as illustrated in Figure 4, with an area of size 457.52 m × 604.02 m, which is partitioned into 10 distinct zones. Without loss of generality, IMDs are assumed to be uniformly distributed within each zone, and the number of IMDs in each cluster is randomly selected from the set $\{5,10,15,20,25,30\}$. Each UAV is initially located at $(0,0,0)$. By explicitly modeling the three-dimensional structure of the environment and specifying feasible flight regions, the UAV is allowed to hover at a fixed altitude $H_{\mathrm{hov}}=10$ m for communication and task offloading. The default simulation parameters, including $H_{\mathrm{hov}}$, are summarized in Table 2.

Figure 4. Digital-twin scenario based on San Francisco.

Table 2. Simulation settings.

Notation	Value	Notation	Value
T	$1800\mathrm{s}$	$H_{\mathrm{hov}}$	$10\mathrm{m}$
$v_{\max }$	$18\mathrm{m}/\mathrm{s}$	B	$20\mathrm{MHz}$
$h_0$	$-50\mathrm{dB}$	${\sigma }^2$	2.53 × ${10}^{-13}\mathrm{W}$
$f^c$	$8\mathrm{GHz}$	$f_k^{\mathrm{loc}}$	$[1,2]\mathrm{GHz}$
${\gamma }^c$	${10}^{-28}$	${\gamma }_k^{\mathrm{loc}}$	${10}^{-28}$
$L_k$	$[0.5,1.5]\mathrm{Mbit}$	$C_k$	$[500,1500]\mathrm{cycles}/\mathrm{bit}$
$O_k$	$0.5$	$t_k^{\max }$	$[0.1,6]\mathrm{s}$
$P_{k,\mathrm{ul}}$	$0.05\mathrm{W}$	$P_{u,\mathrm{dl}}$	$1\mathrm{W}$
$P_{u,\mathrm{ul}}$	$0.5\mathrm{W}$	$P_{k,\mathrm{dl}}$	$0.2\mathrm{W}$
$P_0$	$99.52\mathrm{W}$	$P_1$	$121.64\mathrm{W}$
$P_{\mathrm{hov}}$	$221.16\mathrm{W}$	$U_{\mathrm{tip}}$	$120\mathrm{m}/\mathrm{s}$
$v_0$	$0.002\mathrm{m}/\mathrm{s}$	$d_0$	$0.48$
$s_0$	$0.0001$	$\rho$	$1.225\mathrm{kg}/{\mathrm{m}}^3$
$\delta$	$0.503$

All methods are implemented in PyTorch 2.1.0 with CUDA 12.1 and evaluated on a single NVIDIA GeForce RTX 3090 GPU. Each model is run 50 times with different random seeds, and we report the mean and sample standard deviation of the performance metrics. For the synchronous continuous-control paradigm, we benchmark MADDPG [35], MAPPO [30], and IPPO [33]. For the synchronous discrete-control paradigm, we benchmark MAPPO and IPPO. For the asynchronous discrete-action paradigm, we adopt the state-of-the-art ACAC [38] as the baseline and report its performance on our task.

5.1. Metrics

Our goal is to jointly optimize multi-UAV routing, hovering, and binary task offloading to enhance both service quality and energy efficiency. We let $x_k\in \{0,1\}$ represent the offloading decision for task $I_k$, where $x_k=1$ denotes offloading and $x_k=0$ denotes local execution.

The end-to-end latency of task k is defined as

$$t_k\triangleq (1-x_k)t_k^{\mathrm{loc}}+x_k{t}_k^{\mathrm{mec}}$$

(22)

and each task must satisfy the latency constraint $t_k\le t_k^{\max }$ whenever feasible.

We assess system performance using four metrics: task completion, deadline-normalized latency, UAV energy consumption and an overall efficiency.

5.1.1. Task Completion

We let $\mathbb{I}(·)$ denote the indicator function and let $L_k$ denote the input size (in bits) of task $I_k$. We denote by $L_k^T\in [0,L_k]$ the number of task bits that are successfully processed and returned by the deadline $t_k^{\max }$.

The task-level completion ratio is defined as

$${\psi }_{\mathrm{task}}\triangleq {\displaystyle \frac{1}{K}}\sum _{k\in \mathcal{K}}\mathbb{I}\left(L_k^T=L_k\right)\in [0,1],$$

(23)

where ${\psi }_{\mathrm{task}}\approx 1$ implies that nearly all tasks are fully completed within their respective deadlines, indicating high service reliability.

To account for varying task sizes and partial completions, we introduce a bit-level completion ratio:

$$\psi \triangleq {\displaystyle \frac{{\sum }_{k\in \mathcal{K}}{L}_k^T}{{\sum }_{j\in \mathcal{K}}{L}_j}}\in [0,1],$$

(24)

which measures the fraction of all task bits that are successfully delivered within deadline. A value of $\psi \approx 1$ indicates that nearly all task bits are delivered within deadline, ensuring high throughput under task heterogeneity and imperfect computation.

5.1.2. Deadline-Normalized Latency

For each task k, we let $a_k$ and $c_k$ denote its activation and completion times, respectively, and define the response time as $d_k\triangleq c_k-a_k$. If task k is not completed within the observation horizon, $d_k$ is measured up to the end of the horizon. Since tasks may have different activation times, periods, task sizes, and deadline requirements, their absolute response times are not directly comparable. We therefore quantify timeliness using the following deadline-normalized aggregate score:

$$\mathcal{T}\triangleq {\displaystyle \frac{1}{K}}\sum _{k\in \mathcal{K}}{\displaystyle \frac{d_k}{t_k^{\max }}},$$

(25)

where smaller values of $\mathcal{T}$ indicate better timeliness. In particular, $\mathcal{T}<1$ means that, on average, tasks consume less than their deadline budgets, whereas larger values indicate worse latency performance relative to task deadlines.

5.1.3. UAV Energy Consumption

For each UAV $u\in \mathcal{U}$, we let $E_{u,\mathrm{act}}$ denote propulsion energy (for flight and hovering), and $E_{u,\mathrm{comm}}$ and $E_{u,\mathrm{comp}}$ denote the communication and computing energies, respectively. The total energy consumption of UAV u is

$$E_u\triangleq E_{u,\mathrm{act}}+E_{u,\mathrm{comm}}+E_{u,\mathrm{comp}}.$$

(26)

We define the normalized UAV energy index as

$$\xi \triangleq {\displaystyle \frac{1}{U}}\sum _{u\in \mathcal{U}}{\displaystyle \frac{E_u}{E_u^0}}\in [0,\infty ),$$

(27)

where $E_u^0$ is the initial energy reserve for UAV u, $\xi \approx 0$ indicates minimal energy depletion (high endurance), and $\xi \approx 1$ suggests that, on average, each UAV has consumed the equivalent of a full battery.

5.1.4. Overall Efficiency

Finally, we define an overall dimensionless efficiency index that captures the trade-off between throughput, fairness, timeliness, and energy usage:

$$\lambda \triangleq {\displaystyle \frac{\psi }{\xi ·\mathcal{T}}}\in [0,\infty ).$$

(28)

A larger $\lambda$ indicates better overall efficiency only when interpreted together with $\psi$, $\mathcal{T}$, and $\xi$. Since very small $\xi$ may lead to disproportionately large $\lambda$, we use $\lambda$ as a comparative composite indicator rather than a standalone metric.

5.2. Results and Analysis

5.2.1. Comparison Between Continuous and Discrete Control Paradigms

To evaluate the impact of action space design on UAV-assisted task offloading, we conducted comparative experiments under a scenario with two UAVs and 174 IMDs, where 10 IMDs are selected as cluster heads (CHs). For the discrete-control paradigm, we benchmark MAPPO and IPPO. For the continuous-control paradigm, we benchmark MADDPG, MAPPO, and IPPO. The evaluation includes reward convergence curves, quantitative performance metrics, and UAV trajectory visualizations.

Figure 5 illustrates the reward evolution during training. It can be observed that under continuous action control, the representative methods fail to obtain satisfactory policies in this benchmark. In particular, both MAPPO and IPPO exhibit severe convergence difficulties and fail to achieve stable policies. MADDPG, although designed for continuous control, also does not translate into meaningful performance gains under the realistic 3D collision-avoidance constraints considered here. In contrast, when discrete action control is adopted, both MAPPO and IPPO converge reliably and attain significantly higher long-term rewards, demonstrating improved training stability and effectiveness.

Figure 5. Reward comparison between discrete and continuous control paradigms under MADDPG, MAPPO and IPPO.

The UAV trajectories are visualized in Figure 6. Under continuous control, the UAVs frequently collide with surrounding buildings, which leads to mission failure. This behavior indicates that learning fine-grained continuous actions in complex urban environments with dense obstacles is highly challenging. By contrast, under discrete control, the UAVs move along predefined feasible navigation corridors, strictly remaining within the collision-free flight region. This structured motion enables successful task offloading and robust mission completion.

Figure 6. UAV trajectory visualization under continuous and discrete control paradigms.

Table 3 summarizes the quantitative performance metrics, including task completion ratio $\psi$, task latency $\mathcal{T}$, energy consumption $\xi$, and overall efficiency $\lambda$. All values are reported as the mean and sample standard deviation over multiple independent runs. The results show that discrete control is substantially more effective in this benchmark: both MAPPO and IPPO under discrete control achieve much higher task completion ratios and lower task latency than the continuous-control baselines. By contrast, MADDPG, MAPPO, and IPPO under continuous control all perform poorly, suggesting that the main difficulty lies in the continuous-action formulation under realistic 3D collision-avoidance constraints rather than in a specific MARL optimizer. Although continuous control yields much lower energy consumption, this is mainly because many episodes terminate early due to UAV collisions; therefore, the large $\lambda$ values, especially for MADDPG, should not be interpreted as true performance gains but rather as artifacts of premature mission failure.

Table 3. Performance comparison between continuous- and discrete control paradigms. All results are reported as mean ± sample standard deviation over multiple independent runs. ↑ denotes higher-is-better and ↓ denotes lower-is-better.

Method	$\mathit{\psi }$ ↑	$\mathcal{T}$ ↓	$\mathit{\xi }$ ↓	$\mathit{\lambda }$ ↑
MAPPO (Discrete)	0.5480 ± 0.05	0.5223 ± 0.07	0.6084 ± 0.08	1.7244 ± 0.06
IPPO (Discrete)	0.5479 ± 0.04	0.4915 ± 0.04	0.6268 ± 0.04	1.7782 ± 0.07
MADDPG (Continuous)	0.2327 ± 0.03	0.7080 ± 0.03	0.0120 ± 0.16	26.4316 ± 0.53
MAPPO (Continuous)	0.2327 ± 0.04	0.7081 ± 0.04	0.0481 ± 0.05	6.8278 ± 0.06
IPPO (Continuous)	0.2327 ± 0.04	0.7081 ± 0.04	0.0470 ± 0.04	6.9877 ± 0.06

From a statistical perspective, the discrete control methods also exhibit relatively moderate dispersion across independent runs, indicating stable training outcomes once convergence is achieved. By contrast, the continuous control setting, especially MADDPG, shows noticeably larger fluctuation in some metrics such as $\xi$ and $\lambda$, which further reflects the instability induced by collision-prone behavior and unreliable policy learning in complex urban flight environments.

5.2.2. Comparison Between Synchronous and Asynchronous Decision Paradigms

We further compare synchronous and asynchronous decision-making paradigms in a scenario with two UAVs and 174 IMDs, among which 10 IMDs are selected as cluster heads (CHs). Since the continuous control paradigm exhibits poor performance, only discrete action control methods are considered in this comparison. As the synchronous and asynchronous approaches employ similar discrete motion primitives, trajectory visualizations are omitted. Instead, we focus on the UAV waiting overhead induced by control decisions, reward convergence behavior, and quantitative performance metrics.

Table 4 reports the average waiting time caused by control synchronization. It can be observed that the asynchronous method significantly reduces the waiting overhead compared with synchronous MARL approaches. Although an asynchronous MacDec-POMDP formulation is adopted, for training efficiency and practical simulation considerations, policy decisions and reward settlement are discretized using a fixed step size $\Delta t$. As a result, macro-actions that terminate in continuous time may wait until the next decision epoch, incurring a bounded waiting overhead. This does not violate the asynchronous nature of the framework, as macro-actions are still executed in an event-driven manner and different agents complete their actions at different times; the waiting arises solely from time discretization. Moreover, the waiting overhead in the asynchronous method can be further reduced by decreasing $\Delta t$ at the cost of slower training speed.

Table 4. Average waiting time comparison between synchronous and asynchronous decision paradigms. All results are reported as mean ± sample standard deviation over multiple independent runs. ↓ indicates lower is better.

Method	Average Waiting Time ↓
MAPPO (Synchronous)	172.6282 ± 97.39
IPPO (Synchronous)	161.0173 ± 24.11
ACAC (Asynchronous)	99.2711 ± 62.24

The reported standard deviations also provide useful insight into temporal consistency. In particular, ACAC exhibits a larger spread than IPPO, indicating that the waiting overhead under asynchronous execution is more sensitive to the specific interaction patterns among macro-actions, while its mean value remains substantially lower than those of the synchronous baselines.

Figure 7 illustrates the reward convergence curves for synchronous and asynchronous methods. It can be seen that synchronous MARL approaches converge faster in terms of training iterations, while the asynchronous method requires more iterations to converge but ultimately achieves a higher reward level, indicating superior long-term performance.

Figure 7. Reward comparison between synchronous and asynchronous decision paradigms.

The quantitative performance metrics are summarized in Table 5. The metric results further demonstrate the advantage of asynchronous decision-making. The asynchronous method achieves a higher task completion ratio, lower task latency, and improved overall energy efficiency, confirming that eliminating unnecessary synchronization-induced waiting is beneficial for UAV-assisted task offloading systems. In addition, the standard deviations remain at a relatively comparable level across the three methods, suggesting that the performance gain of ACAC is not obtained at the expense of excessive run-to-run instability.

Table 5. Performance comparison between synchronous and asynchronous decision paradigms. All results are reported as mean ± sample standard deviation over multiple independent runs. ↑ denotes higher-is-better and ↓ denotes lower-is-better.

Method	$\mathit{\psi }$ ↑	$\mathcal{T}$ ↓	$\mathit{\xi }$ ↓	$\mathit{\lambda }$ ↑
MAPPO (Synchronous)	0.5480 ± 0.05	0.5223 ± 0.07	0.6084 ± 0.08	1.7244 ± 0.06
IPPO (Synchronous)	0.5479 ± 0.04	0.4915 ± 0.04	0.6268 ± 0.04	1.7782 ± 0.07
ACAC (Asynchronous)	0.6752 ± 0.03	0.4047 ± 0.05	0.7185 ± 0.04	2.3219 ± 0.05

5.2.3. Impact of the Number of UAVs

We evaluate the impact of the number of UAVs under a moderate IMD density, where 174 IMDs are grouped into 10 clusters with corresponding IMD cluster heads. Table 6 reports the performance metrics for two, three, and four deployed UAVs.

Table 6. Impact of the number of UAVs when the UAV count varies from 2 to 4. All results are reported as mean ± sample standard deviation over multiple independent runs. ↑ denotes higher-is-better and ↓ denotes lower-is-better.

Method	UAVs	$\mathit{\psi }$ ↑	$\mathcal{T}$ ↓	$\mathit{\xi }$ ↓	$\mathit{\lambda }$ ↑
MAPPO (sync)	2	0.5480 ± 0.05	0.5223 ± 0.07	0.6084 ± 0.08	1.7244 ± 0.06
IPPO (sync)	2	0.5479 ± 0.04	0.4915 ± 0.04	0.6268 ± 0.04	1.7782 ± 0.07
ACAC (async)	2	0.6752 ± 0.03	0.4047 ± 0.05	0.7185 ± 0.04	2.3219 ± 0.05
MAPPO (sync)	3	0.6211 ± 0.06	0.4284 ± 0.07	0.5766 ± 0.08	2.5138 ± 0.07
IPPO (sync)	3	0.5803 ± 0.04	0.4398 ± 0.05	0.6270 ± 0.04	2.1048 ± 0.05
ACAC (async)	3	0.7379 ± 0.05	0.3828 ± 0.04	0.7617 ± 0.04	2.5310 ± 0.04
MAPPO (sync)	4	0.7350 ± 0.06	0.4123 ± 0.06	0.5412 ± 0.07	3.2938 ± 0.06
IPPO (sync)	4	0.6280 ± 0.03	0.4376 ± 0.04	0.6116 ± 0.04	2.3466 ± 0.06
ACAC (async)	4	0.8154 ± 0.04	0.3648 ± 0.05	0.7320 ± 0.04	3.0538 ± 0.05

Overall, increasing the number of UAVs consistently improves system performance across all methods, as reflected by higher task completion ratios $\psi$, lower latency $\mathcal{T}$, and higher overall efficiency $\lambda$. This trend indicates that UAV-assisted MEC systems can benefit from increased aerial resource provisioning. Under the same number of UAVs, the asynchronous ACAC method consistently outperforms the synchronous MAPPO and IPPO baselines. Moreover, the performance gap becomes more pronounced as the UAV count increases, highlighting the advantage of asynchronous macro-action decision-making in handling heterogeneous task completion times and de-synchronized agent behaviors. In addition, the standard deviations remain within a limited range across different UAV counts, suggesting that the observed performance trends are reasonably consistent across independent runs. In particular, ACAC exhibits relatively small variations in $\psi$ and $\lambda$ as the number of UAVs increases, indicating robust performance under larger UAV team sizes.

5.2.4. Impact of the Number of IMDs

We next investigate scalability with respect to ground task intensity by varying the number of IMDs while keeping the aerial resources fixed. Specifically, we deploy four UAVs and compare a moderate-density scenario with 174 IMDs grouped into 10 clusters and a high-density scenario with 352 IMDs grouped into 20 clusters, each with corresponding cluster heads. The resulting metrics are summarized in Table 7.

Table 7. Impact of IMD density with four UAVs. Den: IMD density, m: moderate density, h: high density. All results are reported as mean ± sample standard deviation over multiple independent runs. ↑ denotes higher-is-better and ↓ denotes lower-is-better.

Method	Den	$\mathit{\psi }$ ↑	$\mathcal{T}$ ↓	$\mathit{\xi }$ ↓	$\mathit{\lambda }$ ↑
MAPPO (sync)	m	0.7350 ± 0.06	0.4123 ± 0.06	0.5412 ± 0.07	3.2938 ± 0.06
IPPO (sync)	m	0.6280 ± 0.03	0.4376 ± 0.04	0.6116 ± 0.04	2.3466 ± 0.06
ACAC (async)	m	0.8154 ± 0.04	0.3648 ± 0.05	0.7320 ± 0.04	3.0538 ± 0.05
MAPPO (sync)	h	0.6217 ± 0.06	0.4268 ± 0.07	0.5732 ± 0.07	2.5413 ± 0.07
IPPO (sync)	h	0.6313 ± 0.04	0.4256 ± 0.04	0.5349 ± 0.05	2.7728 ± 0.07
ACAC (async)	h	0.6184 ± 0.05	0.4421 ± 0.04	0.7855 ± 0.05	1.7807 ± 0.05

As expected, increasing the IMD density enlarges the total task volume and intensifies contention for communication and computing resources, which generally reduces the task completion ratio $\psi$ and degrades the overall efficiency $\lambda$. For synchronous MAPPO, $\psi$ decreases from $0.7350$ to $0.6217$ and $\lambda$ decreases from $3.2938$ to $2.5413$ as the IMD count approximately doubles, while the latency T remains close, changing from $0.4123$ to $0.4268$. These results indicate that the synchronous baselines preserve reasonable service timeliness under heavier load, but experience a noticeable loss in completion and aggregate utility due to increased congestion.

Among the three methods, synchronous IPPO exhibits the strongest robustness in the high-density regime. When moving from moderate to high density, IPPO slightly improves $\psi$ from $0.6280$ to $0.6313$, reduces T from $0.4376$ to $0.4256$, and decreases $\xi$ from $0.6116$ to $0.5349$, leading to an increase of $\lambda$ from $2.3466$ to $2.7728$. This behavior is consistent with the fact that independent policy learning can be less sensitive to the non-stationarity induced by a larger population of interacting IMDs and more volatile service demand.

For asynchronous ACAC, the moderate-density setting yields the best completion ratio, with $\psi =0.8154$ and $T=0.3648$, demonstrating strong coordination when the environment dynamics are relatively tractable. However, in the high-density scenario ACAC shows a marked reduction in $\psi$ from $0.8154$ to $0.6184$ and in $\lambda$ from $3.0538$ to $1.7807$, accompanied by a higher $\xi$ of $0.7855$. This suggests that the denser and more rapidly changing environment increases the difficulty of stabilizing asynchronous macro-action learning, where heterogeneous task completion times and delayed feedback can exacerbate credit assignment uncertainty. Overall, all methods remain functional as IMD density increases, while the relative advantage shifts toward IPPO in the most congested setting.

From the variance perspective, the standard deviations across moderate- and high-density settings remain within a comparable range for most metrics, indicating that the change in mean performance is the dominant effect. Notably, ACAC does not exhibit a dramatic increase in dispersion under high density, implying that its degradation mainly stems from the increased problem difficulty rather than from a complete loss of training reliability.

5.2.5. Impact of the Flight Speed of UAVs

We further investigate the impact of the UAV flight speed, which is an important system parameter affecting mobility efficiency and service responsiveness. Specifically, we consider a moderate-density scenario with 174 IMDs grouped into 10 clusters and deploy three UAVs to compare different flight-speed settings. The resulting performance metrics are summarized in Table 8.

Table 8. Impact of UAV flight speed with three UAVs. Spe: UAV flight speed (m/s). All results are reported as mean ± sample standard deviation over multiple independent runs. ↑ denotes higher-is-better and ↓ denotes lower-is-better.

Method	Spe	$\mathit{\psi }$ ↑	$\mathcal{T}$ ↓	$\mathit{\xi }$ ↓	$\mathit{\lambda }$ ↑
MAPPO (sync)	10	0.6636 ± 0.06	0.3664 ± 0.07	0.5142 ± 0.09	3.5221 ± 0.09
IPPO (sync)	10	0.6586 ± 0.04	0.3914 ± 0.05	0.5200 ± 0.04	3.2358 ± 0.05
ACAC (async)	10	0.6894 ± 0.04	0.4846 ± 0.04	0.6516 ± 0.04	2.1833 ± 0.05
MAPPO (sync)	18	0.6211 ± 0.06	0.4284 ± 0.07	0.5766 ± 0.08	2.5138 ± 0.07
IPPO (sync)	18	0.5803 ± 0.04	0.4398 ± 0.05	0.6270 ± 0.04	2.1048 ± 0.05
ACAC (async)	18	0.7379 ± 0.05	0.3828 ± 0.04	0.7617 ± 0.04	2.5310 ± 0.04

As the UAV flight speed increases from 10 m/s to 18 m/s, the asynchronous ACAC method exhibits the expected performance trend: the higher speed allows UAVs to reach service locations more quickly, thereby reducing deadline-normalized latency from $0.4846$ to $0.3828$ and improving the task completion ratio from $0.6894$ to $0.7379$. Although the normalized energy index $\xi$ increases from $0.6516$ to $0.7617$, reflecting the higher mobility cost associated with faster movement, the gain in task completion and timeliness is sufficient to improve the overall efficiency $\lambda$ from $2.1833$ to $2.5310$. By contrast, the two synchronous methods do not benefit from the higher flight speed in the same way. For both MAPPO and IPPO, increasing the speed from 10 m/s to 18 m/s leads to lower task completion ratios and worse latency, together with higher energy consumption. One possible explanation is that, in the synchronous setting, UAVs must jointly handle mobility control and task-offloading decisions under periodically arriving tasks, while their actions are still coupled through slot-level synchronization. As a result, the potential advantage of faster movement cannot always be converted into better effective service, because the learning problem becomes more challenging and the coordination inefficiency induced by synchronization may offset the mobility gain. The standard deviations in Table 8 remain moderate across different speed settings, indicating that the observed performance differences are reasonably consistent over independent runs.

6. Discussion

While the proposed benchmark and comparative study provide a practical and systematic view of MARL for UAV-assisted MEC under realistic 3D collision avoidance constraints, several limitations remain. First, we assume that each UAV can serve at most one device at a time. In practice, enabling concurrent service to multiple devices may improve offloading success and overall efficiency, but it requires explicit modeling and allocation of communication power, interference, and onboard edge computation resources. Second, the benchmark adopts a simplified LoS-dominant communication model. Although this assumption improves tractability and allows the comparison to focus on the coupling among UAV mobility, task offloading, and MARL decisin-making, more realistic wireless effects such as NLoS propagation, blockage, shadowing, interference, or even real-world channel distributions may significantly affect transmission reliability and latency, and therefore should be incorporated in future extensions. Third, although tasks are periodic over time, agent observations are primarily local and instantaneous and do not explicitly exploit the temporal regularities of periodic task arrivals. Incorporating task state modeling and prediction that captures periodic patterns may enable more anticipatory navigation and scheduling decisions under strict deadlines. Fourth, the benchmark adopts a binary offloading model, and allowing partial offloading and task partitioning may provide more flexible trade-offs among latency, energy, and completion probability.

Several directions remain worthy of further investigation. When multiple UAVs serve devices in the same region, interference and resource contention become inevitable, motivating interference-aware coordination and resource allocation. This also enables extending the learning setting beyond fully independent or fully cooperative agents toward competitive and mixed interaction models. Scalability is another key concern, since directly scaling the current framework to large fleets may lead to prohibitive complexity, suggesting the need for hierarchical architectures that combine regional coordination with local control. In addition, incorporating energy replenishment mechanisms such as charging and battery swapping can support longer-term service. Finally, considering heterogeneous UAV capabilities can better match real deployments, which we leave for future work.

7. Conclusions

This paper benchmarks multi-agent reinforcement learning for UAV-assisted mobile edge computing under realistic 3D collision avoidance navigation constraints for deadline-constrained periodic task offloading. By modeling feasible mobility via a collision-free 3D navigation graph with shortest-path execution and evaluating three representative MARL paradigms under a unified protocol, we reveal clear trade-offs in offloading success, end-to-end latency, and energy consumption induced by practical 3D constraints. The benchmark and insights aim to guide the design of reliable UAV–MEC systems in dense urban environments and to support reproducible future research in realistic 3D settings.