MA-PF-AD3PG: A Multi-Agent DRL Algorithm for Latency Minimization and Fairness Optimization in 6G IoV-Oriented UAV-Assisted MEC Systems

Highlights

What are the main findings?

• We develop a priority–fairness coupled optimization framework together with a multi-agent DRL algorithm (MA-PF-AD3PG) to jointly optimize latency, fairness, and task priority in UAV-assisted 6G IoV MEC systems.
• The proposed algorithm incorporates an occlusion-aware dynamic deadline model, fairness-aware preprocessing, and an adaptive delayed update mechanism, achieving significantly improved convergence stability and scheduling performance.

What are the implication of the main findings?

• The results demonstrate that fairness-driven multi-UAV cooperation can sustain near-perfect service fairness while reducing latency under dynamic vehicular environments.
• The findings offer practical insights for designing next-generation UAV-assisted drone communication systems that require balanced QoS, priority awareness, and system-wide efficiency.

Abstract

The rapid proliferation of connected and autonomous vehicles in the 6G era demands ultra-reliable and low-latency computation with intelligent resource coordination. Unmanned Aerial Vehicle (UAV)-assisted Mobile Edge Computing (MEC) provides a flexible and scalable solution to extend coverage and enhance offloading efficiency for dynamic Internet of Vehicles (IoV) environments. However, jointly optimizing task latency, user fairness, and service priority under time-varying channel conditions remains a fundamental challenge.To address this issue, this paper proposes a novel Multi-Agent Priority-based Fairness Adaptive Delayed Deep Deterministic Policy Gradient (MA-PF-AD3PG) algorithm for UAV-assisted MEC systems. An occlusion-aware dynamic deadline model is first established to capture real-time link blockage and channel fading. Based on this model, a priority–fairness coupled optimization framework is formulated to jointly minimize overall latency and balance service fairness across heterogeneous vehicular tasks. To efficiently solve this NP-hard problem, the proposed MA-PF-AD3PG integrates fairness-aware service preprocessing and an adaptive delayed update mechanism within a multi-agent deep reinforcement learning structure, enabling decentralized yet coordinated UAV decision-making. Extensive simulations demonstrate that MA-PF-AD3PG achieves superior convergence stability, 13–57% higher total rewards, up to 46% lower delay, and nearly perfect fairness compared with state-of-the-art Deep Reinforcement Learning (DRL) and heuristic methods.

1. Introduction

The continuous emergence of new services driven by the Internet of Things (IoT), artificial intelligence, and the industrial Internet has placed unprecedented multi-dimensional demands on network communication, characterized by the requirements of high bandwidth, low latency, and high reliability [1]. These requirements are particularly critical in the Internet of Vehicles (IoV)—one of the most promising IoT application scenarios—where dynamic interactions among vehicles, infrastructure, and users impose stringent performance constraints on communication networks [2].

As the next-generation mobile communication technology, the Sixth Generation (6G) network is envisioned to provide ultra-high data rates, enhanced reliability, and extremely low latency [3]. These capabilities align naturally with the performance needs of emerging IoV services, making 6G a key enabler for real-time vehicular data transmission and edge-assisted computation [4]. In parallel, Mobile Edge Computing (MEC)—a foundational component of 6G—brings computation resources closer to end users, thereby alleviating the congestion and transmission delays inherent to conventional cloud computing architectures. This proximity reduces end-device energy consumption, prolongs battery life, and enhances the overall Quality of Service (QoS) [5]. In IoV environments, the synergy between 6G and MEC is particularly vital for supporting latency-sensitive applications such as autonomous driving, traffic monitoring, and safety-critical control [6,7].

However, conventional two-dimensional ground network infrastructures often suffer from inadequate coverage in complex urban environments due to physical obstructions and terrain irregularities [8,9]. To overcome these limitations, 6G is evolving toward three-dimensional network architectures by integrating Unmanned Aerial Vehicles (UAVs) as aerial base stations [10]. UAVs can effectively extend coverage in obstructed urban areas or post-disaster regions, ensuring reliable communication links for autonomous vehicles and rescue systems. Consequently, UAVs are increasingly regarded as indispensable components of IoV-oriented MEC systems, providing flexible and on-demand communication and computation support [11,12].

The Mixed Broadband Reliable Low-Latency Communication (MBRLLC) service introduced in 6G integrates the advantages of enhanced Mobile Broadband (eMBB) and Ultra-Reliable Low-Latency Communication (URLLC). This service model inherently reflects task prioritization in IoV applications: eMBB tasks (e.g., in-vehicle 4K video streaming) emphasize bandwidth; URLLC tasks (e.g., collision avoidance) demand millisecond-level latency and the highest priority; and MBRLLC tasks (e.g., augmented reality navigation) require balanced performance and occupy a medium priority level. Such heterogeneous priorities call for adaptive and priority-aware task offloading strategies in MEC systems. Neglecting task priorities may delay safety-critical services and compromise vehicular safety [13,14].

Nevertheless, achieving efficient and fair task offloading in UAV-assisted IoV-MEC systems remains highly challenging. The high mobility of vehicles results in rapidly changing network topology, rendering static offloading strategies ineffective. Meanwhile, UAV-assisted coverage is inherently vulnerable to dynamic occlusions, mobility-induced channel fluctuations, and strict energy constraints, which may lead to intermittent connectivity and degraded reliability compared with terrestrial infrastructures. Moreover, the use of MEC along vehicular routes introduces non-stationary performance, as vehicles continuously traverse heterogeneous coverage regions with time-varying resource availability and traffic demand. Such performance variability may cause fluctuating latency and throughput, potentially degrading QoS if not properly addressed. These challenges highlight the necessity of adaptive and robustness-aware resource allocation mechanisms that jointly optimize latency, fairness, and service priority in dynamic 6G IoV environments.

2. Related Work

UAV-assisted MEC has been extensively studied as a means to provide flexible computation support and reduce service latency in dynamic IoV environments.

In [15], the CLACMO framework was developed to jointly optimize caching, UAV trajectory, and offloading policies by integrating a variational autoencoder with Deep Reinforcement Learning (DRL). This approach achieved significant improvements in average reward and task completion rate through efficient latent-space exploration. Meanwhile, a DRL-based model for dynamic UAV trajectory control and user association was presented in [16], enabling adaptive decision-making under time-varying network conditions and improving task offloading efficiency in single-UAV scenarios. To further enhance fairness, Yang et al. [17] designed a Reward-Shaping DRL algorithm based on a max–min optimization framework, which simultaneously optimized UAV trajectories, task allocation, and service scheduling in post-disaster edge environments.

Building upon these efforts, a federated multi-agent reinforcement learning framework was introduced in [18] to minimize the age of information in UAV-assisted MEC systems modeled as partially observable Markov decision processes (POMDPs). The design enabled collaborative policy learning among UAVs to improve information freshness. In addition, ref. [19] proposed a hybrid algorithm that combines particle swarm optimization (PSO) diversity with a twin delayed deep deterministic policy gradient (TD3) model, effectively balancing offloading, energy consumption, and UAV trajectory planning. Similarly, Zheng et al. [20] introduced a priority-aware DRL scheme that jointly planned UAV access paths and user offloading strategies, thereby enhancing system utility under heterogeneous task priorities.

On another front, the authors in [21] examined multi-UAV cooperation and resource allocation, where a joint power–computation optimization algorithm was designed to minimize total energy consumption of IoT terminals while maintaining service reliability. Furthermore, Huang et al. [22] leveraged federated DRL to optimize UAV caching and vehicular offloading in IoV networks, improving cache hit rates and reducing redundant transmissions. Following a similar direction, ref. [23] proposed a hierarchical air–ground MEC architecture combined with dual-timescale DRL to optimize UAV trajectories and reduce information latency. Finally, Liu et al. [24] developed a QoS-aware multi-agent DDPG framework for vehicular edge computing, coordinating offloading and resource allocation among multiple vehicles to improve overall system throughput and QoS satisfaction.

While these studies have advanced UAV-assisted MEC from the perspectives of trajectory optimization, resource scheduling, and distributed decision-making, most existing approaches still mainly focus on improving offloading efficiency, but do not jointly address heterogeneous service priorities and fairness among vehicular users under dynamic network conditions. Furthermore, transient link variations and limited UAV energy and computation capacity make real-time coordination more complex in large-scale IoV scenarios. This motivates the development of a unified framework that adaptively allocates resources while balancing priority and fairness.

The main contributions of this paper are summarized as follows:

• Occlusion-Aware Dynamic Deadline Model: A novel environment-aware model is developed to capture the time-varying impact of occlusion on channel conditions, transforming real-time link variations into dynamic noise parameters. The model incorporates system-wide hard deadlines based on total task volume to better reflect vehicular service constraints.
• Priority–Fairness Coupled Optimization Framework: The latency minimization problem is reformulated into a two-level optimization structure, consisting of fairness-driven vehicular scheduling and UAV trajectory/resource allocation. This formulation enables adaptive decision-making while maintaining fairness and priority consistency among heterogeneous tasks.
• Multi-Agent Priority-based Fairness Adaptive Delayed DDPG (MA-PF-AD3PG) Algorithm: A novel MADRL algorithm is proposed by embedding a three-dimensional state representation—priority evaluation, connection history, and fairness factor—within an enhanced MADDPG architecture. The algorithm achieves adaptive trajectory control and offloading optimization, improving system fairness and reducing latency in dynamic 6G IoV environments.

This paper is organized as follows. Section 2 provides a comprehensive review of related studies on IoV and UAV-assisted MEC task offloading. Section 3 describes the system model for the considered IoV scenario and formulates the corresponding optimization problem. Section 4 models the problem as a MDP and presents a MADRL-based algorithm for dynamic resource allocation. Section 5 evaluates the proposed approach through extensive comparative experiments to demonstrate its effectiveness and stability. Finally, Section 6 concludes the paper and discusses potential directions for future research.

3. System Model

We consider a UAV-assisted MEC system deployed in a 6G IoV scenario, as illustrated in Figure 1. The system consists of M UAVs and N vehicular end devices (VEs) distributed within a two-dimensional region of size $L\times W$. Each UAV flies at a fixed altitude H, is equipped with limited onboard computing resources, and provides edge computation services to vehicles through wireless communication links. Let $\mathcal{M}=\{1,2,\dots ,M\}$ and $\mathcal{N}=\{1,2,\dots ,N\}$ denote the sets of UAVs and VEs, respectively. The system operates in discrete time, where the total operation duration $T_{\mathrm{tot}}$ is divided into T equal-length time slots with duration $T_{\mathrm{slot}}$. The time slot index set is written as $\mathcal{T}=\{1,2,\dots ,T\}$.

In each slot $t\in \mathcal{T}$, ${\mathrm{VE}}_n$ generates a computation task with data size $D_n\left(t\right)\in [D_{n,min},D_{n,max}]$. Due to limited local processing capability, each task can be partially executed locally and partially offloaded to UAVs for remote computation. After the offloaded task is processed, the result is transmitted back to the vehicle through the downlink channel. To meet diverse QoS requirements in IoV applications, tasks are classified into three priority categories: URLLC (high priority), MBRLLC (medium priority), and eMBB (low priority).

UAV mobility and energy constraints are considered to ensure reliable and safe operation. The distance between any two UAVs must remain above a predefined threshold $d_{\mathrm{safe}}$ for collision avoidance. The remaining battery energy of ${\mathrm{UAV}}_m$, denoted $E_{m,\mathrm{re}}\left(t\right)$, must satisfy $E_{m,\mathrm{re}}\left(t\right)\ge E_{\mathrm{th}}$ to ensure continuous flight and computation. Additionally, ${\mathrm{UAV}}_m$ must process a minimum total task volume $D_m$ within $T_{\mathrm{tot}}$ to maintain balanced load utilization.

3.1. Communication Model

The position of ${\mathrm{VE}}_n$ at time slot t is denoted by

$${\mathbf{Q}}_n\left(t\right)=[X_n\left(t\right),Y_n\left(t\right),0],$$

(1)

while ${\mathrm{UAV}}_m$ flies at a fixed altitude H, and its position is represented as

$${\mathbf{Q}}_m\left(t\right)=[X_m\left(t\right),Y_m\left(t\right),H].$$

(2)

Accordingly, the Euclidean distance between ${\mathrm{UAV}}_m$ and ${\mathrm{VE}}_n$ is given by

$$d_{m,n}\left(t\right)=\sqrt{{(X_m\left(t\right)-X_n\left(t\right))}^2+{(Y_m\left(t\right)-Y_n\left(t\right))}^2+H^2}.$$

(3)

${\mathrm{UAV}}_m$ moves with velocity $v_m\left(t\right)$ and heading angle ${\theta }_m\left(t\right)$ in each slot, and its position is updated as

$$\begin{array}{cc}\hfill X_m(t+1)& =X_m\left(t\right)+v_m\left(t\right)T_{\mathrm{fly}}cos{\theta }_m\left(t\right),\hfill \\ \hfill Y_m(t+1)& =Y_m\left(t\right)+v_m\left(t\right)T_{\mathrm{fly}}sin{\theta }_m\left(t\right),\hfill \end{array}$$

(4)

where $0\le v_m\left(t\right)\le v_{max}$ and $0\le {\theta }_m\left(t\right)<2\pi$.

We denote by $C_{m,n}\left(t\right)\in \{0,1\}$ the association variable indicating whether ${\mathrm{VE}}_n$ is connected to ${\mathrm{UAV}}_m$ at slot t.

In dense urban IoV environments, wireless links between UAVs and VEs are highly susceptible to dynamic occlusions caused by buildings, roadside infrastructures, and vehicle mobility. To capture this effect, we introduce the binary occlusion indicator $N_{m,n}\left(t\right)$, where $N_{m,n}\left(t\right)=0$ denotes a line-of-sight (LoS) condition and $N_{m,n}\left(t\right)=1$ represents a non-line-of-sight (NLoS) condition. Rather than modeling occlusion as an abstract event, $N_{m,n}\left(t\right)$ is explicitly coupled with the physical channel condition by directly determining the effective noise power in the achievable rate expression, i.e., $P_{\mathrm{LoS}}$ for LoS links and $P_{\mathrm{NLoS}}$ for occluded links with $P_{\mathrm{NLoS}}>P_{\mathrm{LoS}}$.

The uplink transmission rate from ${\mathrm{VE}}_n$ to ${\mathrm{UAV}}_m$ is given by

$$R_{m,n}\left(t\right)=C_{m,n}\left(t\right)B{log}_2\left(1+{\displaystyle \frac{P_{n,\mathrm{tx}}{h}_{m,n}^2\left(t\right)d_{m,n}^{-\alpha }\left(t\right)}{(1-N_{m,n}\left(t\right))P_{\mathrm{LOS}}+N_{m,n}\left(t\right)P_{\mathrm{NLOS}}}}\right),$$

(5)

where B is the channel bandwidth, $P_{n,\mathrm{tx}}$ is the transmit power of ${\mathrm{VE}}_n$, $h_{m,n}\left(t\right)$ denotes small-scale fading, and $\alpha$ is the path-loss exponent.

This modeling approach enables transient link blockages to be translated into instantaneous throughput degradation, which in turn affects task transmission delay and the feasibility of meeting slot-level execution deadlines. In this sense, the proposed occlusion-aware model induces an implicit dynamic deadline tightening effect: when a link becomes occluded, the reduced transmission rate increases the risk of deadline violation within a fixed time slot, especially for high-priority latency-sensitive tasks. By embedding occlusion effects directly into the rate and latency formulations, the proposed framework captures realistic channel uncertainty without introducing additional heuristic constraints.

3.2. Computation Model

In task offloading scheduling, the fraction of tasks offloaded from ${\mathrm{VE}}_n$ to ${\mathrm{UAV}}_m$ at slot t is defined by the offloading ratio $w_{m,n}\left(t\right)$ ($0\le w_{m,n}\left(t\right)\le 1$), which is dynamically adjusted with ${\mathrm{UAV}}_m$’s flight angle ${\theta }_m\left(t\right)$ and velocity $v_m\left(t\right)$ to improve system offloading efficiency.

3.2.1. Latency

The total task execution latency of ${\mathrm{VE}}_n$ comprises three components: local computing latency, transmission latency, and edge computing latency.

The local computing latency of ${\mathrm{VE}}_n$ at time slot t depends on the locally processed data volume, the required CPU cycles per unit data, and the local CPU processing capacity:

$$T_{n,\mathrm{local}}\left(t\right)=\sum _{m=1}^M\left({\displaystyle \frac{D_n\left(t\right)\left(1-w_{m,n}\left(t\right)\right)C_{\mathrm{VE}}}{f_n}}\right)C_{m,n}\left(t\right),\forall n\in \mathcal{N},t\in \mathcal{T},$$

(6)

where $D_n\left(t\right)$ is the task data volume for ${\mathrm{VE}}_n$, $C_{\mathrm{VE}}$ represents the CPU cycles required per unit of task for each VE, and $f_n$ is the computing capacity of ${\mathrm{VE}}_n$.

The transmission latency depends on the portion of data offloaded to ${\mathrm{UAV}}_m$ and the corresponding uplink transmission rate between ${\mathrm{VE}}_n$ and ${\mathrm{UAV}}_m$ at time slot t:

$$T_{n,\mathrm{trans}}\left(t\right)=\sum _{m=1}^M\left({\displaystyle \frac{D_n\left(t\right)w_{m,n}\left(t\right)}{R_{m,n}\left(t\right)}}\right)C_{m,n}\left(t\right),\forall n\in \mathcal{N},t\in \mathcal{T}.$$

(7)

The edge computing latency reflects the time required by ${\mathrm{UAV}}_m$ to execute the offloaded task portion, considering the computing capacity of ${\mathrm{UAV}}_m$ and the total number of VEs served by ${\mathrm{UAV}}_m$ in the current slot:

$$T_{n,\mathrm{edge}}\left(t\right)=\sum _{m=1}^M\left({\displaystyle \frac{D_n\left(t\right)w_{m,n}\left(t\right)C_{\mathrm{UAV}}}{f_m{\sum }_{i=1}^N{C}_{m,i}\left(t\right)}}\right)C_{m,n}\left(t\right),\forall n\in \mathcal{N},t\in \mathcal{T},$$

(8)

where $f_m$ represents the computing capacity of ${\mathrm{UAV}}_m$, and $C_{\mathrm{UAV}}$ represents the CPU cycles required per unit of data for the UAV.

Since each ${\mathrm{VE}}_n$ may either execute tasks locally or offload them for remote processing, its total execution latency is determined by the slower of the two computation paths:

$$T_{\mathrm{VE},n}\left(t\right)=max\left(T_{n,\mathrm{local}}\left(t\right),T_{n,\mathrm{trans}}\left(t\right)+T_{n,\mathrm{edge}}\left(t\right)\right).$$

(9)

Correspondingly, the service latency of ${\mathrm{UAV}}_m$ is the maximum latency among all VEs served by ${\mathrm{UAV}}_m$ at time slot t:

$$T_{\mathrm{UAV},m}\left(t\right)=\underset{1\le n\le N}{max}{C}_{m,n}\left(t\right)T_{\mathrm{VE},n}\left(t\right).$$

(10)

3.2.2. Energy Consumption

The total system energy consumption consists of the energy consumed by both VEs and UAVs.

For ${\mathrm{VE}}_n$, the transmission energy consumption is given by

$$E_{n,\mathrm{trans}}\left(t\right)=P_{n,\mathrm{tx}}{T}_{n,\mathrm{trans}}\left(t\right).$$

(11)

The local computing energy consumption, influenced by chip architecture, is

$$E_{n,\mathrm{local}}\left(t\right)=k{f}_n^2\left(1-w_{m,n}\left(t\right)\right)D_n\left(t\right)C_{\mathrm{VE}},$$

(12)

where k is a constant factor related to the chip architecture.

Thus, the total energy consumption of ${\mathrm{VE}}_n$ is:

$$E_{\mathrm{VE},n}\left(t\right)=E_{n,\mathrm{local}}\left(t\right)+E_{n,\mathrm{trans}}\left(t\right).$$

(13)

For ${\mathrm{UAV}}_m$, the total energy consumption consists of three components: flight energy, hovering energy, and edge computing energy. The flight energy consumption is:

$$E_{m,\mathrm{fly}}\left(t\right)={\displaystyle \frac{M_{m}g{t}_{\mathrm{fly}}{v}_m\left(t\right)}{K}},$$

(14)

where $M_m$ is the mass of ${\mathrm{UAV}}_m$, g is the gravitational acceleration, $t_{\mathrm{fly}}$ is the flight time, and K is the lift-to-drag ratio, which characterizes the UAV’s aerodynamic performance.

The hovering energy consumption of ${\mathrm{UAV}}_m$ at time slot t is:

$$E_{m,\mathrm{hover}}\left(t\right)=P_m{t}_{\mathrm{hover}},$$

(15)

where $P_m$ is the hovering power and $t_{\mathrm{hover}}$ is the hovering time.

The edge computing energy consumption for ${\mathrm{UAV}}_m$ is:

$$E_{m,\mathrm{edge}}\left(t\right)=\sum _{n=1}^N{C}_{m,n}\left(t\right)k{f}_{\mathrm{UAV}}^2{w}_{m,n}\left(t\right)D_n\left(t\right)C_{\mathrm{UAV}}.$$

(16)

Therefore, the total energy consumption of ${\mathrm{UAV}}_m$ is:

$$E_{\mathrm{UAV},m}\left(t\right)=E_{m,\mathrm{fly}}\left(t\right)+E_{m,\mathrm{edge}}\left(t\right)+E_{m,\mathrm{hover}}\left(t\right).$$

(17)

3.3. Task Prioritization

To address the heterogeneous QoS requirements of three typical 6G vehicular services (eMBB, URLLC, and MBRLLC), the system adopts a dual mechanism combining fixed priority quantification and reward-penalty incentives. This framework ensures preferential scheduling of high-priority tasks, with priority levels tied to service characteristics: URLLC tasks (e.g., emergency collision avoidance warnings, safety-critical) have the highest priority; MBRLLC tasks (e.g., AR navigation rendering, bandwidth-latency balanced) are secondary; eMBB tasks (e.g., in-vehicle 4K entertainment, latency-tolerant non-critical) are lowest. To implement this framework, the priority of tasks generated by ${\mathrm{VE}}_n$ at time slot t (denoted ${\mathrm{Pr}}_{\mathrm{VE},n}\left(t\right)$) is quantized solely by service type, formulated as:

$${\mathrm{Pr}}_{\mathrm{VE},n}\left(t\right)=\left\{\begin{array}{cc}{\mathrm{Pr}}_{\mathrm{URLLC}}\hfill & \mathrm{if}\mathrm{the}\mathrm{task}\mathrm{belongs}\mathrm{to}\mathrm{URLLC};\hfill \\ {\mathrm{Pr}}_{\mathrm{MBRLLC}}\hfill & \mathrm{if}\mathrm{the}\mathrm{task}\mathrm{belongs}\mathrm{to}\mathrm{MBRLLC};\hfill \\ {\mathrm{Pr}}_{\mathrm{eMBB}}\hfill & \mathrm{if}\mathrm{the}\mathrm{task}\mathrm{belongs}\mathrm{to}\mathrm{eMBB};\hfill \end{array}\right.$$

where ${\mathrm{Pr}}_{\mathrm{URLLC}}>{\mathrm{Pr}}_{\mathrm{MBRLLC}}>{\mathrm{Pr}}_{\mathrm{eMBB}}$, establishing discrete hierarchical priority tiers.

To enforce these priority tiers in scheduling, the system integrates a reward-penalty mechanism: when ${\mathrm{UAV}}_m$ serves ${\mathrm{VE}}_n$, a positive reward is given if the task finishes within $T_{\mathrm{slot}}$ (i.e., $T_{\mathrm{VE},n}\left(t\right)\le T_{\mathrm{slot}}$), and a negative penalty for delays. Formally, ${\mathrm{UAV}}_m$’s priority-based reward at slot t is defined as:

$${\mathrm{Pr}}_{\mathrm{UAV},m}\left(t\right)=\sum _{n=1}^N{C}_{m,n}\left(t\right){\mathrm{Pr}}_{\mathrm{VE},n}\left(t\right)\mathbb{I}\left(T_{\mathrm{VE},n}\left(t\right)\le T_{\mathrm{slot}}\right),$$

(18)

where $\mathbb{I}\left(x\right)$ is the indicator function, $\mathbb{I}\left(x\right)=1$ if the task is completed on time and $\mathbb{I}\left(x\right)=-1$ if the task is delayed.

The discrete priority levels are introduced as an abstraction for tractable modeling and stable learning. Since the proposed framework is not restricted by priority granularity, it can be readily extended to continuous or context-dependent QoS representations.

3.4. User Fairness

In UAV-assisted MEC systems, ensuring user fairness involves avoiding the excessive allocation of resources to specific users while maintaining timely service for high-priority tasks, and preventing low-priority users from extended service deprivation. A fairness mechanism that balances “priority orientation” and “opportunity equilibrium” is formulated below.

3.4.1. Key Variable Definitions

Two foundational variables quantify service allocation dynamics:

• Service availability variable $s_n\left(t\right)$: This binary variable indicates whether ${\mathrm{VE}}_n$ is served ($s_n\left(t\right)=1$ if served, $s_n\left(t\right)=0$ if not). It is derived from the connection variable $C_{m,n}\left(t\right)$:

  $$s_n\left(t\right)=\sum _{m=1}^M{C}_{m,n}\left(t\right)$$

  (19)
• Cumulative service count $S_n\left(t\right)$: This variable represents the total number of times ${\mathrm{VE}}_n$ has been served up to time slot t, capturing the history of service allocation:

  $$S_n\left(t\right)=\sum _{\tau =1}^t{s}_n\left(\tau \right)$$

  (20)

3.4.2. Modeling of Target Service Count

To align resource allocation with task priorities, ${\mathrm{VE}}_n$’s target service count at slot t, denoted $S_n^{*}\left(t\right)$, represents the cumulative service it should ideally receive based on its priority. This is computed by proportionally distributing the system’s total cumulative services according to the priority of each VE:

$$S_n^{*}\left(t\right)={\displaystyle \frac{{\mathrm{Pr}}_{\mathrm{VE},n}\left(t\right)}{{\sum }_{i=1}^N{\mathrm{Pr}}_{\mathrm{VE},i}\left(t\right)}}\sum _{i=1}^N{S}_i\left(t\right)$$

(21)

where ${\sum }_{i=1}^N{S}_i\left(t\right)$ is the system’s total cumulative services up to slot t, and ${\displaystyle \frac{{\mathrm{Pr}}_{\mathrm{VE},n}\left(t\right)}{{\sum }_{i=1}^N{\mathrm{Pr}}_{\mathrm{VE},i}\left(t\right)}}$ is ${\mathrm{VE}}_n$’s priority share. This ensures that higher-priority users have higher target service counts, such that if ${\mathrm{VE}}_A$ has double the priority of ${\mathrm{VE}}_B$, then $S_A^{*}\left(t\right)\approx 2S_B^{*}\left(t\right)$.

3.4.3. Fairness Evaluation Metrics

To quantify system fairness, two metrics are introduced:

• Deviation variance $\mathrm{Var}\left(t\right)$: This metric measures the mean squared deviation between each user’s actual cumulative service count and its target service count, reflecting how well the system aligns with priority goals:

  $$\mathrm{Var}\left(t\right)={\displaystyle \frac{1}{N}}\sum _{n=1}^N{\left[S_n\left(t\right)-S_n^{*}\left(t\right)\right]}^2$$

  (22)
• Fairness index $\mathrm{Fair}\left(t\right)$: This index intuitively reflects the fairness of the system at time slot t. It is defined as:

  $$\mathrm{Fair}\left(t\right)=1-{\displaystyle \frac{\mathrm{Var}\left(t\right)}{max\left(\mathrm{Var}\left(t\right)\right)}}$$

  (23)

$\mathrm{Fair}\left(t\right)\to 1$ means most users’ actual services match targets (high priority-oriented fairness); $\mathrm{Fair}\left(t\right)\to 0$ indicates severe deviation, leading to over-service or under-service.

3.5. Formulation of the Optimization Problem

The decision variables of the optimization problem include four key components: $\mathbf{C}={\left[C_{m,n}\left(t\right)\right]}_{M\times N\times T}\in {\{0,1\}}^{M\times N\times T}$, which denotes the offloading service selection of ${\mathrm{UAV}}_m$ for ${\mathrm{VE}}_n$; $\mathbf{W}={\left[w_{m,n}\left(t\right)\right]}_{M\times N\times T}\in {[0,1]}^{M\times N\times T}$, representing the proportion of ${\mathrm{VE}}_n$’s tasks offloaded to ${\mathrm{UAV}}_m$; $\mathbf{V}={\left[v_m\left(t\right)\right]}_{M\times T}\in {[0,v_{\max }]}^{M\times T}$, indicating the flight speed of ${\mathrm{UAV}}_m$; and $\Theta ={\left[{\theta }_m\left(t\right)\right]}_{M\times T}\in {[0,2\pi ]}^{M\times T}$, which stands for the flight angle of ${\mathrm{UAV}}_m$.

The objective of the optimization problem is to regulate ${\mathrm{UAV}}_m$’s dynamic scheduling and task offloading strategies, aiming to maximize user fairness and UAV priority rewards while minimizing system latency. The problem is formulated as

$$\begin{array}{cc}\hfill \underset{\mathbf{C},\mathbf{W},\mathbf{V},\Theta }{max}\sum _{t=1}^T(& {\beta }_1\mathrm{Fair}\left(t\right)+{\beta }_2\sum _{m=1}^M{\mathrm{Pr}}_{\mathrm{UAV},m}\left(t\right)-{\beta }_3\sum _{m=1}^M{T}_{\mathrm{UAV},m}\left(t\right))\hfill \end{array}$$

(24)

subject to the following constraints:

• Vehicle positions are confined within the service area: $0\le X_n\left(t\right)\le L,0\le Y_n\left(t\right)\le W,\forall n\in \mathcal{N},t\in \mathcal{T}.$
• UAV flight control constraints: $0\le {\theta }_m\left(t\right)\le 2\pi ,0\le v_m\left(t\right)\le v_{max},\forall m\in \mathcal{M},t\in \mathcal{T}.$
• Task priority constraint: ${\mathrm{Pr}}_{\mathrm{VE},n}\left(t\right)\in \{1,2,3\},\forall n\in \mathcal{N},t\in \mathcal{T}.$
• UAV energy constraint: ${\sum }_{t=1}^T{E}_{\mathrm{UAV},m}\left(t\right)\le E_{\mathrm{UAV},\max },\forall m\in \mathcal{M}.$
• Unique association constraint: ${\sum }_{m=1}^M{C}_{m,n}\left(t\right)\le 1,\forall n\in \mathcal{N},t\in \mathcal{T}.$
• Minimum task completion requirement: ${\sum }_{t=1}^T{\sum }_{n=1}^N{C}_{m,n}\left(t\right)w_{m,n}\left(t\right)D_n\left(t\right)\ge D_m,\forall m\in \mathcal{M}.$
• UAV collision-avoidance constraint: $d_{m,j}\left(t\right)\ge d_{\mathrm{safe}},\forall m,j\in \mathcal{M},t\in \mathcal{T}.$

4. Optimization Framework and Algorithm Development

The formulated joint optimization problem is NP-hard due to the coupling of mixed-integer decision variables, non-convex objectives, and stochastic system dynamics. Task offloading, UAV resource allocation, and priority-aware fairness constraints are tightly interdependent across time slots, preventing decomposition into tractable convex subproblems. While traditional optimization techniques, such as mixed-integer programming or relaxation-based methods, may still be applicable to small-scale or static scenarios, their computational complexity grows prohibitively in large-scale and highly dynamic IoV environments. Consequently, exact or deterministic solutions become impractical.

Therefore, heuristic and learning-based approaches, such as DRL, offer an effective alternative by enabling online decision-making under uncertainty. By modeling the problem as a Markov Decision Process (MDP), the proposed MA-PF-AD3PG framework efficiently approximates high-quality solutions in large-scale and stochastic settings while maintaining scalability and adaptability.

4.1. MDP Formulation

4.1.1. State Space

At each time slot t, the state space captures the real-time environmental information of UAVs and VEs required for the DRL agent’s decision-making. Specifically, for each ${\mathrm{UAV}}_m\in \mathcal{M}$, the state includes the remaining battery energy $E_{m,\mathrm{re}}\left(t\right)$, the current position $Q_m\left(t\right)$, and the remaining task volume $D_{m,\mathrm{re}}\left(t\right)$. For each ${\mathrm{VE}}_n\in \mathcal{N}$, the state comprises the position $Q_n\left(t\right)$, the task priority ${\mathrm{Pr}}_n\left(t\right)$, the task data volume $D_n\left(t\right)$, and the occlusion status $N_{m,n}\left(t\right)$, which indicates whether the communication link between the ${\mathrm{VE}}_n$ and ${\mathrm{UAV}}_m$ is obstructed. The total state space has a dimension of $4M+5N$, which scales linearly with the number of UAVs and VEs. All state parameters are normalized to $[0,1]$ to eliminate scale differences and ensure stable DRL training. At time slot t, the state for ${\mathrm{UAV}}_m$ is represented as:

$$s_m\left(t\right)=\{E_{m,\mathrm{re}}\left(t\right),Q_m\left(t\right),D_{m,\mathrm{re}}\left(t\right),Q_n\left(t\right),{\mathrm{Pr}}_n\left(t\right),D_n\left(t\right),N_{m,n}\left(t\right)\mid 1\le n\le N\}$$

(25)

The total state for all M UAVs at time t is given by the concatenation of the individual UAV states:

$$s\left(t\right)=\{s_1\left(t\right),s_2\left(t\right),\dots ,s_M\left(t\right)\}$$

(26)

4.1.2. Action Space

The action space specifies the decision variables available to the DRL agent under a given state. For each UAV $m\in \mathcal{M}$, the action consists of the offloading decision $C_{m,n}\left(t\right)$, flight angle ${\theta }_m\left(t\right)$, flight speed $v_m\left(t\right)$, and task offloading ratio $w_{m,n}\left(t\right)$, as previously defined. These variables are constrained as follows: $C_{m,n}\left(t\right)\in \{0,1\}$, $0\le {\theta }_m\left(t\right)<2\pi$, $0\le v_m\left(t\right)\le v_{\max }$, and $0\le w_{m,n}\left(t\right)\le 1$. The action for ${\mathrm{UAV}}_m$ is therefore given by:

$$a_m\left(t\right)=\{C_{m,n}\left(t\right),{\theta }_m\left(t\right),v_m\left(t\right),w_{m,n}\left(t\right)\mid 1\le n\le N\}$$

(27)

To satisfy the DRL model’s input requirements, the discrete variable $C_{m,n}\left(t\right)$ is one-hot encoded, and the continuous variables ${\theta }_m\left(t\right)$, $v_m\left(t\right)$, and $w_{m,n}\left(t\right)$ are normalized to $[0,1]$. The overall action for all UAVs at time t is the concatenation of individual UAV actions:

$$a\left(t\right)=\{a_1\left(t\right),a_2\left(t\right),\dots ,a_M\left(t\right)\}$$

(28)

This action representation enables decentralized decision-making and enhances the DRL agent’s ability to efficiently handle the UAV-assisted MEC optimization problem.

4.1.3. Reward Function

The reward function defines the objective signal for the DRL agent and aligns with the optimization goals in Section 3.5. It aims to maximize user fairness, prioritize task rewards, and minimize latency. Penalty terms for collision risk and energy constraints are included to maintain feasible operation. At each time slot t, the reward function is defined as:

$$r\left(t\right)=\mathrm{Fair}\left(t\right)+\sum _{m=1}^M\left[{\beta }_1{\mathrm{Pr}}_{\mathrm{UAV},m}\left(t\right)-{\beta }_2{\mathrm{T}}_{\mathrm{UAV},m}\left(t\right)-{\beta }_3{\mathrm{PE}}_{\mathrm{UAV},m}\left(t\right)\right]-{\beta }_4{\mathrm{PE}}_{\mathrm{col}}\left(t\right),$$

(29)

where $\mathrm{Fair}\left(t\right)$ is the fairness metric calculated at time slot t. ${\mathrm{Pr}}_{\mathrm{UAV},m}\left(t\right)$ represents the priority reward of ${\mathrm{UAV}}_m$. ${\mathrm{T}}_{\mathrm{UAV},m}\left(t\right)$ is the latency associated with ${\mathrm{UAV}}_m$, penalizing delays that may impact the overall system performance. ${\mathrm{PE}}_{\mathrm{UAV},m}\left(t\right)$ denotes the energy consumption penalty for ${\mathrm{UAV}}_m$. ${\mathrm{PE}}_{\mathrm{col}}\left(t\right)$ is the penalty for collision risk, ensuring that the system avoids unsafe UAV positioning. The weighted factors ${\beta }_1,{\beta }_2,{\beta }_3,{\beta }_4$ allow for flexibility in adjusting the relative importance of each of these objectives, ensuring the reward function is tailored to different operational scenarios.

The objective of the DRL agent is to maximize the cumulative reward across the entire time horizon T:

$$\underset{\mathcal{A}}{max}\sum _{t=1}^{T}r\left(t\right).$$

(30)

4.2. The MADDPG Algorithm

Given the system’s multiple autonomous UAVs (multi-agents) and continuous action space, this paper employs the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm to solve the optimization problem. MADDPG extends the single-agent DDPG method by incorporating a centralized training and decentralized execution paradigm. This adaptation is crucial for managing the dynamic interactions among UAVs, aligning with the system’s collaborative scheduling requirements.

4.2.1. Network Architecture

The MADDPG algorithm utilizes a dual-network structure, consisting of an Actor Network and a Critic Network for each UAV, with a shared global Critic Network to evaluate the collective actions of all agents. This design enables efficient credit assignment among UAVs and facilitates collaborative training.

1.

Actor Network and Actor Target Network

Each UAV is equipped with an independent Actor Network, which generates deterministic actions based on the UAV’s local state. The output action for ${\mathrm{UAV}}_m$ at time t is denoted as $a_m\left(t\right)={\pi }_{\psi }\left(s_m\left(t\right)\right),$ where $s_m\left(t\right)$ is the local state of ${\mathrm{UAV}}_m$ at time t, and ${\pi }_{\psi }$ is the policy function parameterized by $\psi$. The Actor Target Network performs the same function but uses the next state $s_m^{\prime }\left(t\right)$ to produce the next target action $a_m^{\prime }\left(t\right)$, such that $a_m^{\prime }\left(t\right)={\pi }_{{\psi }_t}\left(s_m^{\prime }\left(t\right)\right)$ and ${\psi }_t$ represents the parameters of the target network.

2.

Critic Network and Critic Target Network

Unlike the single-agent DDPG, the Critic Network in MADDPG evaluates the effectiveness of the joint actions of all UAVs. It takes the global state $s\left(t\right)$ (which integrates the states of all UAVs and VEs) and the joint action $a\left(t\right)$ as inputs. The Critic outputs the global state-action value function $Q_{\varphi }(s\left(t\right),a\left(t\right))$, which is used to estimate the cumulative expected reward from executing the joint action, and adopts the parameter $\varphi$. The Critic Target Network is updated similarly, using the next global state $s^{\prime }\left(t\right)$ and next joint action $a^{\prime }\left(t\right)$ as inputs, and outputs the next target value $Q_{{\varphi }_t}(s^{\prime }\left(t\right),a^{\prime }\left(t\right))$, where ${\varphi }_t$ denotes the parameters of the target network.

4.2.2. Experience Replay

To mitigate the issue of temporal correlations in reinforcement learning data, MADDPG employs a replay memory $\mathcal{R}$, where the agent stores past experiences as tuples of the form $(s\left(t\right),a\left(t\right),r\left(t\right),s^{\prime }\left(t\right),a^{\prime }\left(t\right))$. During training, a mini-batch of experiences is sampled from the replay buffer to update the network parameters. This mechanism stabilizes training by allowing the model to learn from past experiences, rather than being solely reliant on immediate transitions.

4.2.3. Network Parameter Update

The updates for the MADDPG algorithm involve three key components: the Actor Network, the Critic Network, and their corresponding Target Networks.

1.

Critic Network Update

The Critic Network is updated by minimizing the Temporal Difference (TD) error for the sampled experiences:

$${\delta }_i=r_i+\gamma ·Q_{{\varphi }_t}(s_i^{\prime },a_i^{\prime })-Q_{\varphi }(s_i,a_i)$$

(31)

where $\gamma \in [0,1]$ is the discount factor. The Critic’s parameters $\varphi$ are updated by gradient descent to minimize the TD error.

2.

Actor Network Update

The Actor Network is updated using the policy gradient theorem. The objective is to maximize the expected cumulative reward $J\left({\pi }_{\psi }\right)$ by adjusting the Actor’s parameters $\psi$ as follows:

$${\nabla }_{\psi }J\left({\pi }_{\psi }\right)\approx {\displaystyle \frac{1}{M}}\sum _{i=1}^M\left[{\left.{\nabla }_{a_m}{Q}_{\varphi }(s_i,a_i)\right|}_{a_m={\pi }_{\psi }\left(s_{m,i}\right)}·{\nabla }_{\psi }{\pi }_{\psi }\left(s_{m,i}\right)\right]$$

(32)

3.

Soft Update of Target Networks

To stabilize training and avoid overfitting to recent experiences, MADDPG applies soft updates to the Target Networks:

$${\psi }_t\leftarrow (1-\tau ){\psi }_t+\tau \psi ,{\varphi }_t\leftarrow (1-\tau ){\varphi }_t+\tau \varphi$$

(33)

where $\tau$ is the soft update coefficient, typically set to a small value (e.g., $\tau =0.001$).

4.3. The MA-PF-AD3PG Algorithm

We propose the Multi-Agent Priority-based Fairness Adaptive Delayed Update DDPG (MA-PF-AD3PG) algorithm to handle hybrid-variable optimization, reduce training oscillations, and accelerate convergence in multi-UAV MEC environments. The proposed algorithm enhances MADDPG by integrating discrete variable preprocessing and a delayed update mechanism, improving task scheduling efficiency and overall learning stability.

4.3.1. Discrete Service Selection Preprocessing

While MADDPG operates on continuous actions, the offloading decision variable $C_{m,n}\left(t\right)$ is discrete and cannot be directly optimized within this framework. To address this, we propose a preprocessing step that enables the UAV to select appropriate VEs as service objects based on task priority and fairness considerations.

To prevent service imbalance among VEs, the preprocessing method introduces a fairness weighting mechanism. This mechanism assigns higher selection weights to VEs with fewer connection counts than the average. The fairness weight for each ${\mathrm{VE}}_n$ served by ${\mathrm{UAV}}_m$ at time t is defined as:

$$P_{m,n}\left(t\right)=\left(\sum _{n=1}^J{\mathrm{Pr}}_{\mathrm{VE},n}\left(t\right)\right){\left(1-{\displaystyle \frac{S_n\left(t\right)}{\overline{S}\left(t\right)}}\right)}^{F_f},$$

(34)

where ${\sum }_{n=1}^J{\mathrm{Pr}}_{\mathrm{VE},n}\left(t\right)$ denotes the total weight of VEs with the same priority, $S_n\left(t\right)$ represents the cumulative connection count for ${\mathrm{VE}}_n$, and $\overline{S}\left(t\right)$ is the average connection count across all VEs. The fairness factor $F_f$ is a positive constant that controls the strength of fairness in service selection.

Leveraging the previously defined $\mathrm{Fair}\left(t\right)$ metric, Algorithm 1 presents a dynamic connection allocation mechanism for the vehicular network. A UAV-VE connection probability matrix is generated, embedding fairness constraints to reflect real-time service demands and historical connection patterns. Each UAV then performs probabilistic selection based on this matrix to complete VE pairing, and a conflict detection module ensures that each VE connects to at most one UAV.

This preprocessing converts the original MILP-based problem into a continuous form, simplifying optimization. The original action space of dimension $2M+2MN$, derived from UAV flight, connection, and offloading variables, is reduced to $2M+MN$ after removing the discrete term $C_{m,n}\left(t\right)$, significantly decreasing computational complexity.

Algorithm 1 Fairness-Guaranteed Scheduling Algorithm (FGSA)

Require: Task set $\mathcal{N}$; Priority mapping rule; Fairness factor $F_f$; Time horizon T. Ensure: Service object selection result $\mathrm{SelectedVE}\left(t\right)$.       1: Initialize priority benchmarks ${\mathrm{Pr}}_{\mathrm{VE},n}\left(t\right)\in \{{\mathrm{Pr}}_{\mathrm{URLLC}},{\mathrm{Pr}}_{\mathrm{MBRLLC}},{\mathrm{Pr}}_{\mathrm{eMBB}}\}$; Service count variables $s_n\left(t\right),S_n\left(t\right)$; Fairness threshold ${\theta }_{\mathrm{Fair}}$:       2: for $t=1$ to T do       3:      Calculate fairness metrics: Compute target cumulative service count $S_n^{*}\left(t\right)$; Deviation variance $\mathrm{Var}\left(t\right)$; and Fairness metric $\mathrm{Fair}\left(t\right)$.       4:      if $\mathrm{Fair}\left(t\right)<{\theta }_{\mathrm{Fair}}$ then       5:             Trigger scheduling strategy adjustment;       6:      end if       7:      for each UAV $m\in \mathcal{M}$ do       8:             Probabilistic service selection: Calculate the selection weight $P_{m,n}\left(t\right)$ via (34).       9:             Determine $\mathrm{SelectedVE}\left(t\right)$ by probabilistic sampling (e.g., roulette wheel selection), ensuring each VE connects to at most one UAV.     10:      end for     11:      for each VE $n\in \mathcal{N}$ do     12:             State update and iteration: Update the instantaneous service count $s_n\left(t\right)$ and the cumulative service count $S_n\left(t\right)$.     13:      end for     14: end for

4.3.2. Network Delayed Update Mechanism

The MA-PF-AD3PG algorithm employs a delayed update mechanism controlled by parameter $\delta$, which determines the update frequency of the Actor and Critic networks. This delay mitigates training oscillations and stabilizes convergence. The delay factor is typically set to $\delta =2$, implying network updates every two time steps. This choice represents a practical trade-off between training stability and responsiveness in dynamic UAV-assisted MEC environments. Specifically, updating the Actor at every timestep ($\delta =1$) may lead to parameter oscillations due to rapidly changing value estimates, while larger delays ($\delta \ge 3$) slow down policy adaptation to fast-varying network conditions.

Under this mechanism, the Critic main network is updated at every timestep to provide accurate and timely value estimation, whereas the Actor main network and the corresponding target networks are updated with a delay of $\delta =2$. This asymmetric update strategy stabilizes multi-agent training while preserving sufficient adaptability to dynamic system states, making it well suited to the considered UAV–MEC scenario.

4.3.3. MA-PF-AD3PG Algorithm Overview

Building upon the decentralized execution and centralized training paradigm of MADDPG, the proposed MA-PF-AD3PG algorithm integrates the fairness-aware preprocessing and the delayed update mechanism introduced in Section 4.3.1 and Section 4.3.2, respectively. Each UAV’s Actor network generates local actions based on its observed state, while a shared Critic network evaluates the global state–action value using these actions and fairness-aware system states. This joint evaluation enables coordinated policy optimization among UAVs.

The proposed framework adapts efficiently to dynamic and large-scale UAV interactions through fairness-driven task assignment and adaptive policy learning. The overall workflow of the algorithm is summarized in Algorithm 2.

Algorithm 2 MA-PF-AD3PG Training Procedure

Require: System parameters $M,N,T,D,B,f_{\mathrm{UAV}},f_{\mathrm{VE}},E_{max},t_{\mathrm{fly}},T_{\mathrm{tot}},T_{\mathrm{slot}},v_{\max },P_{\mathrm{LoS}},P_{\mathrm{NLoS}}$ Ensure: Optimized policies $\{C,w,v,\theta \}$       1: Initialization. Set hyperparameters: discount factor $\gamma$; replay buffer $\mathcal{R}$; batch size; learning rates ${\alpha }_{\psi }$ and ${\alpha }_{\varphi }$; soft update coefficient $\tau$; action exploration noise $\epsilon$; delayed update parameter $\delta$; initialize network parameters $\psi ,{\psi }_t,\varphi ,{\varphi }_t$;       2: for each episode $i\in \mathrm{range}(\max \_\mathrm{episodes})$ do       3:       Reset environment: UAV and VE positions $Q_m$, $Q_n$, remaining energy $E_{\mathrm{rem},m}=E_{\max }$, remaining task load $D_{\mathrm{rem},m}=D$, and $\mathrm{Fair}\left(t\right)=0$;       4:       for each time slot $t\in \mathrm{range}\left(T\right)$ do       5:             for each agent $\in \mathrm{range}\left(M\right)$ do       6:                   Calculate occlusion status $N_{m,n}\left(t\right)$ based on $Q_m\left(t\right),Q_n\left(t\right)$       7:                   Observe current state $s_m\left(t\right)$ and obtain $C_{m,n}\left(t\right)$ via Algorithm 1;       8:                   Actor outputs action $a_m\left(t\right)={\pi }_{\psi }\left(s_m\left(t\right)\right)+ϵ$;       9:                   Execute action; update UAV states: $E_{\mathrm{rem},m}(t+1)$, $D_{\mathrm{rem},m}(t+1)$, $Q_m(t+1)$;     10:                   Observe the next state $s_m(t+1)$ and the next action $a_m(t+1)$     11:                   Update $E_{\mathrm{rem},m}(t+1)$, $D_{\mathrm{rem},m}(t+1)$, and $Q_m(t+1)$ via kinematic model;     12:                   Observe the next state $s_m(t+1)$; generate next action $a_m(t+1)$;     13:                   if $D_{\mathrm{rem},m}(t+1)==0$ then     14:                         Set delay $=0$; terminate the inner loop (time step t);     15:                   else if UAV exceeds boundary then     16:                         Set $v\left(t\right)=0$ (halt UAV movement); add position violation penalty $P{E}_{\mathrm{pos}}$;     17:                   else if $E_{\mathrm{rem},m}(t+1)<0$ then     18:                         Set $w_n\left(t\right)=0$ (stop task offloading); add energy violation penalty $P{E}_{\mathrm{en}}$;     19:                   else     20:                         Calculate the service delay $T_{\mathrm{UAV},m}\left(t\right)$ using (10);     21:                   end if     22:             end for     23:             Calculate fairness metric $\mathrm{Fair}\left(t\right)$ using (23);     24:             Calculate priority reward: ${\mathrm{Pr}}_{\mathrm{reward}}\left(t\right)={\beta }_1{\sum }_{m=1}^M{\mathrm{Pr}}_{\mathrm{UAV},m}\left(t\right)$;     25:             Calculate latency penalty: ${\mathrm{Lat}}_{\mathrm{penalty}}\left(t\right)={\beta }_2{\sum }_{m=1}^M{T}_{\mathrm{UAV},m}\left(t\right)$;     26:             Calculate violation penalty: ${\mathrm{Viol}}_{\mathrm{penalty}}\left(t\right)={\beta }_{3}P{E}_{\mathrm{en}}+{\beta }_{4}P{E}_{\mathrm{pos}}$;     27:             Total reward: $r\left(t\right)=\mathrm{Fair}\left(t\right)+{\mathrm{Pr}}_{\mathrm{reward}}\left(t\right)-{\mathrm{Lat}}_{\mathrm{penalty}}\left(t\right)-{\mathrm{Viol}}_{\mathrm{penalty}}\left(t\right)$;     28:             Store transition $\left(s\right(t),a(t),r(t),s(t+1\left)\right)$ in replay buffer $\mathcal{R}$;     29:             Update Critic network parameter $\varphi$;     30:             if $tmod\delta ==0$ then     31:                   Update Actor network parameter $\psi$;     32:                   Soft-update target network parameters: ${\psi }_t$, ${\varphi }_t$;     33:             end if     34:       end for     35: end for

As shown in Figure 2, the discrete preprocessing reduces the action-space dimensionality, improving computational efficiency. The delayed update mechanism further stabilizes training by mitigating gradient oscillations in the reduced space. The collaborative multi-agent structure improves coordination among UAVs. Collectively, these components improve stability and fairness while reducing computational complexity.

4.3.4. Time Complexity of MA-PF-AD3PG

Let E denote the total number of training episodes, T the number of time steps per episode, B the batch size, $\delta$ the delayed update interval, and $D_{\mathrm{Actor}}$ and $D_{\mathrm{Critic}}$ the computational complexities of the Actor and Critic networks. Since network structures remain fixed, both can be approximated as a constant K.

Auxiliary operations (e.g., UAV state updates and reward computation) require $\mathcal{O}\left(M\right)$ time per step, negligible compared with network computation. The Critic network updates at each step with complexity $\mathcal{O}\left(B{D}_{\mathrm{Critic}}\right)$, while the Actor and Target networks update every $\delta$ steps with complexity $\mathcal{O}\left(B{D}_{\mathrm{Actor}}\right)$. Hence, the per-episode cost is: $\mathcal{O}(TB(D_{\mathrm{Critic}}+{\displaystyle \frac{D_{\mathrm{Actor}}}{\delta }}))\approx \mathcal{O}\left(TBK\right),$ and the overall training complexity is: $\mathcal{O}\left(ETB\right).$

Compared with conventional MADDPG $\mathcal{O}\left(ETB(D_{\mathrm{Critic}}+D_{\mathrm{Actor}})\right)$, MA-PF-AD3PG reduces Actor updates by a factor of $\delta$, maintaining the same asymptotic order $\mathcal{O}\left(ETB\right)$ with lower constant overhead.

5. Simulation Analysis

To evaluate the performance of the proposed MA-PF-AD3PG algorithm in UAV-assisted MEC systems for IoV scenarios, extensive simulations are conducted within a planar area of $100\mathrm{m}\times 100\mathrm{m}$. All UAVs operate at a fixed altitude of 100 m. Two UAVs are deployed, initially located at coordinates $[50,50,100]$ and $[75,75,100]$ (unit: m), respectively. The total system bandwidth is set to 1 MHz. Detailed environmental parameters are summarized in

Table 1. Environmental Simulation Parameters.

Parameter	Value	Unit
Noise Power (LOS)	−100	dB
Noise Power (NLOS)	−80	dB
VE Computing Frequency	0.2	GHz
UAV Computing Frequency	1.2	GHz
CPU Cycles per bit	1000	Cycles
Uplink Transmission Power	0.1	W
Reference Channel Gain (1 m)	−50	dB
UAV Weight	9.65	kg
UAV Battery Capacity	500	kJ
Maximum UAV Flight Speed	20	m/s

.

Regarding the network structure of MA-PF-AD3PG, the Actor network consists of three fully connected (FC) layers with 300, 400, and 300 neurons, respectively, followed by an output layer corresponding to the action dimension. The Critic network adopts the same FC configuration as the Actor, but its output layer contains a single neuron without activation to directly output Q-values. The number of training iterations is set to 1000, and a minimum mean-square-error (MSE) constraint is applied to regulate the exploration rate. The basic hyperparameters are summarized in

Table 2. Neural Network HyperParameters.

Parameter	Value
Maximum Episodes ($\max \_\mathrm{episode}$)	1000
Actor Network Learning Rate (${\alpha }_{\psi }$)	0.001
Critic Network Learning Rate (${\alpha }_{\varphi }$)	0.002
Discount Factor ($\gamma$)	0.5
Soft Update Coefficient ($\tau$)	0.01
Minimum Variance (${\mathrm{var}}_{\min }$)	0.01
Replay Buffer Size ($\mathcal{R}$)	10,000
Batch Size (B)	64
Delayed Update Parameter ($\delta$)	2

.

5.1. Validation of the Priority-Based Fairness Mechanism

Jain’s fairness index is adopted to quantify the fairness of resource allocation among users, which is defined as

$$J={\displaystyle \frac{{\left({\sum }_{n=1}^N{x}_n\right)}^2}{N{\sum }_{n=1}^N{x}_n^2}},$$

(35)

where $x_n$ denotes the achieved service metric of ${\mathrm{VE}}_n$. The index ranges from $1/N$ to 1, with larger values indicating more equitable resource distribution.

To verify the effectiveness of the proposed priority-based fairness mechanism, three multi-agent learning strategies are evaluated under a unified AD3PG framework. The proposed algorithm, termed MA-PF-AD3PG, incorporates priority-aware fairness into the decision-making process. Two baseline algorithms are considered for comparison: MA-RA-AD3PG, where UAVs randomly select serving VEs, and MA-PR-AD3PG, which schedules VEs solely based on task priority without explicit fairness consideration. In the simulation, two UAVs and eight total VEs are considered with a total task volume of 100 Mbits. The performance comparison results are illustrated in Figure 3 and Figure 4.

Figure 3 compares the total reward performance of the three algorithms, where a higher reward indicates better overall system performance. The results show that MA-PF-AD3PG achieves the highest average reward, outperforming MA-PR-AD3PG and MA-RA-AD3PG by 14.42% and 26.40%, respectively. This demonstrates that incorporating priority-aware fairness significantly improves resource utilization efficiency.

Figure 4 presents the comparison of Jain’s fairness index. The proposed MA-PF-AD3PG achieves an average Jain index of 0.999, with the box plot tightly concentrated around 0.9988, indicating near-perfect fairness. In contrast, MA-RA-AD3PG and MA-PR-AD3PG achieve average fairness indices of 0.968 and 0.895, respectively, with the latter falling below the commonly accepted “high fairness” threshold of 0.9. These results confirm that priority-only scheduling may lead to severe service imbalance. Overall, MA-PF-AD3PG provides an optimal trade-off between system efficiency and fairness while maintaining stable convergence behavior.

5.2. Impact of the Delayed Update Mechanism

To evaluate the role of the delayed update parameter in improving convergence stability and optimization efficiency, MA-PF-AD3PG (with delayed update) is compared with MA-PF-DDPG (without delayed update) under the same setup. As shown in Figure 5, MA-PF-AD3PG achieves a significantly higher converged total reward (approximately $-20$ versus $-80$ for MA-PF-DDPG), indicating a substantially improved reward ceiling and better task performance after convergence. While both algorithms reach a minimum penalty of 0, MA-PF-AD3PG exhibits a lower average delay (2.20 vs. 2.55), confirming its advantage in delay control. These results verify that the delayed update mechanism enhances both convergence stability and overall optimization performance.

5.3. Comparison with Other DRL Algorithms

To further demonstrate the superiority of MA-PF-AD3PG, it is compared with several representative DRL algorithms under the same MA-PF framework.

DQN [25]: Value-based framework mapping state-action pairs to Q-values, quantifying expected cumulative rewards.

AC [26]: Merges value estimation and policy gradients, with experience replay and target networks boosting stability.

TD3 [27]: Advanced DDPG variant with dual Critics, using their minimum output to mitigate value overestimation.

KNN-DDPG [11]: Non-parametric method relying on local neighborhood info for lightweight, training-free classification.

In the same scenario with two UAVs and eight VEs (100 Mbits total task volume), the results in Figure 6 indicate that MA-PF-AD3PG achieves the best total reward after convergence (around $-57$), surpassing MA-PF-AC (around $-360$), MA-PF-KNN, MA-PF-DDPG, and MA-PF-TD3 (around $-130$). Moreover, its training curve is notably smoother, maintaining the optimal reward level throughout the 1000-episode process, which demonstrates excellent learning efficiency and convergence stability in multi-agent task scheduling scenarios.

To further evaluate scalability, we consider a larger-scale setup with 16 VEs (2 high-priority, 4 medium-priority, 10 low-priority) and the same total task volume of 100 Mbits. As shown in Figure 7, although MA-AC and MA-PF-KNN converge faster initially, MA-PF-AD3PG ultimately achieves the highest and most stable reward, converging at around 350 episodes. Its final reward exceeds that of MA-PF-DDPG by over 32%, MA-PF-KNN and MA-PF-AC by more than 27%, and MA-PF-TD3 by nearly 16%, confirming its superior convergence stability in large-scale heterogeneous environments.

To validate the generalization capability of MA-PF-AD3PG under different task volumes, additional simulations are conducted using eight VEs (five low priority, two medium priority, and one high priority). Two scenarios are evaluated: Scenario 1, where each UAV serves one VE, and Scenario 2, where each UAV serves two VEs simultaneously.

In Scenario 1, task volumes are set to 60, 80, 100, 120, and 140 Mbits. As shown in Figure 8, where the upper subfigure is a bar chart with 95% confidence intervals (CI) and the lower one is a trend chart, MA-PF-AD3PG consistently outperforms all comparative algorithms across all task sizes. For instance, at 100 Mbits, its maximum total reward is approximately $-105$, compared to $-120$ for MA-PF-DDPG, reflecting a 13% advantage. The decreasing trend in reward with increasing task volume is attributed to intensified resource contention and latency penalties under higher loads, consistent with practical system behavior. Notably, MA-AC exhibits fluctuations between 80–100 Mbits due to unstable policy convergence caused by imbalanced priority weights.

In Scenario 2, task volumes are 80, 100, 120, 140, and 160 Mbits. As shown in Figure 9, MA-PF-AD3PG not only maintains its superiority but achieves even greater performance gains as each UAV serves more VEs. For example, at 160 Mbits, MA-PF-AD3PG reaches a maximum reward of approximately $-140$, while MA-PF-AC drops to about $-260$, indicating a 46% advantage. The same decreasing trend with increasing load is observed, due to intensified scheduling pressure and latency accumulation.

To further evaluate the algorithm’s scalability under different numbers of VEs, the maximum total reward is analyzed using combined box plots (with 95% CIs) and trend curves, as illustrated in Figure 10 for Scenario 1. As the number of VEs increases from 4 to 12, the proposed MA-PF-AD3PG algorithm consistently achieves the highest maximum total reward among all compared methods. In contrast, MA-PF-DDPG, MA-PF-TD3, and other baselines exhibit significant performance degradation and pronounced fluctuations. For example, when the number of VEs reaches 12, the reward of MA-PF-TD3 decreases to approximately $-68.5$, whereas that of MA-PF-AD3PG remains above $-29.5$, representing nearly a 57% improvement. These results clearly demonstrate the robustness and scalability of the proposed algorithm as the network scale expands.

In Scenario 2, the number of VEs is set to 4, 8, 12, 16, and 20. As shown in Figure 11, the MA-PF-AD3PG algorithm maintains its superior performance, consistently achieving higher rewards than MA-PF-AC, MA-PF-KNN, and MA-PF-TD3 across all configurations. For instance, when the number of VEs increases to 20, MA-PF-AD3PG attains a maximum total reward of approximately $-29$, while MA-PF-TD3 drops to around $-48$, corresponding to a 42% performance gain. Although minor fluctuations occur in some configurations, the overall results confirm that MA-PF-AD3PG demonstrates the best stability and adaptability among all compared algorithms. These findings are consistent with those in Scenario 1, further validating the scalability and robustness of the proposed MA-PF-AD3PG framework in large-scale UAV–MEC resource scheduling.

As the number of VEs increases, the proposed MA-PF-AD3PG maintains smooth convergence and consistent performance advantages over the baseline strategies, indicating that the learning process scales well with larger state and action spaces. Moreover, the decentralized execution with centralized training paradigm enables effective adaptation to dynamic mobility without explicit coordination among UAVs. Overall, these results demonstrate the scalability of the proposed approach and its suitability for larger-scale UAV-assisted IoV deployments.

6. Conclusions

This paper investigates optimization challenges in UAV-assisted MEC systems for 6G IoV scenarios through two main stages. First, a fairness-oriented target selection mechanism is proposed to improve user fairness in UAV service allocation. By incorporating user priority, connection frequency, fairness weighting, and probabilistic selection, the proposed approach effectively balances service fairness and system efficiency. Simulation results show that it reduces service opportunity disparities among VEs while ensuring timely execution of high-priority tasks.

Second, a dynamic multi-UAV MEC model is established considering the characteristics of 6G MBRLLC, eMBB, and URLLC services. To jointly optimize latency, fairness, and task priority, a multi-agent DRL algorithm, termed MA-PF-AD3PG, is developed. By integrating fairness-aware preprocessing and a delayed update mechanism within a centralized training and decentralized execution framework, the proposed method improves convergence stability and computational efficiency. Simulation results demonstrate that MA-PF-AD3PG consistently outperforms baseline algorithms in terms of total reward, latency reduction, and fairness, while exhibiting robust convergence and scalability.

Future work will explore advanced learning techniques, such as federated multi-agent learning and meta-adaptive policy optimization, and extend the proposed framework to multi-tier edge–cloud architectures and dynamic multi-cell UAV networks to further enhance adaptability in large-scale IoV environments.