Trajectory Prediction and Decision Optimization for UAV-Assisted VEC Networks: An Integrated LSTM-TD3 Framework

Abstract

With the rapid development of intelligent transportation systems (ITSs) and Internet of Things (IoT), vehicle-mounted edge computing (VEC) networks are facing the challenge of handling increasingly growing computation-intensive and latency-sensitive tasks. In the UAV-assisted VEC network, by introducing mobile edge servers, the coverage of ground infrastructure is effectively supplemented. However, there is still the problem of decision-making lag in a highly dynamic environment. This paper proposes a deep reinforcement learning framework based on the long short-term memory (LSTM) network for trajectory prediction to optimize resource allocation in UAV-assisted VEC networks. Uniquely integrating vehicle trajectory prediction with the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm, this framework enables proactive computation offloading and UAV trajectory planning. Specifically, we design an LSTM network with an attention mechanism to predict the future trajectory of vehicles and integrate the prediction results into the optimization decision-making process. We propose state smoothing and data augmentation techniques to improve training stability and design a multi-objective optimization model that incorporates the Age of Information (AoI), energy consumption, and resource leasing costs. The simulation results show that compared with existing methods, the method proposed in this paper significantly reduces the total system cost, improves the information freshness, and exhibits better environmental adaptability and convergence performance under various network conditions.

1. Introduction

The accelerated advancement of intelligent transportation systems (ITSs) and Internet of Vehicles (IoV) technologies has stimulated the proliferation of various emerging in-vehicle applications; examples include autonomous driving, real-time navigation, augmented reality, and multimedia content sharing, which are rapidly gaining popularity [1]. These applications are typically computationally intensive and latency-sensitive, posing severe challenges to the response speed and computing power of the network. However, since in-vehicle terminals are usually constrained by limited computing resources, storage capacity, and energy supply, relying solely on local computing often fails to meet the high reliability and low latency requirements of these applications [2].

Vehicular edge computing (VEC), a technology that deploys servers at the network edge to enable vehicle users to offload computing tasks to nearby roadside units (RSUs), has been widely recognized as an effective solution to the above challenges [3]. VEC reduces the computing burden on cellular networks by shortening data transmission distances, offering users more efficient and reliable network services. However, the traditional static deployment of edge servers still faces multiple challenges: first, in remote areas with sparse infrastructure coverage, vehicle users may be unable to access edge computing services; second, in urban congested areas with high vehicle density, limited RSU resources struggle to meet a large number of concurrent computing requests; and third, urban building obstructions may degrade signal quality, affecting the quality of service (QoS) [4].

To address these issues, researchers have proposed a UAV-assisted VEC network architecture by integrating UAVs (Unmanned Aerial Vehicles) into the VEC network [5]. Military use of UAVs has become a widespread practice globally and in civilian fields due to their advantages of easy deployment, high mobility, and strong line-of-sight (LoS) links [6,7]. In VEC networks, UAVs can serve as mobile edge servers to provide additional computing and communication services, thereby relieving the pressure on ground computing and communication resources while filling VEC network coverage gaps. Furthermore, UAV-assisted VEC systems leverage more reliable LoS links to offer flexible on-demand services and respond more promptly to user requests [8].

However, UAV-assisted VEC networks still face complex resource optimization challenges. First, the high-speed mobility of UAVs and vehicles leads to rapid changes in network topology and channel conditions; second, the randomness and burstiness of vehicle tasks make resource requirements difficult to predict; and third, UAV energy constraints severely impact the system’s continuous service capabilities [9,10]. Traditional studies focus on minimizing latency or energy consumption [11,12], or balancing the two [13,14], but rarely consider the critical metric of information timeliness, modeled by Age of Information (AoI).

The integration of AoI is innovative and critical for safety-critical applications, especially autonomous driving. Autonomous vehicles depend on real-time environmental data processed by UAV-assisted VEC nodes for decisions. Stale computation results of pedestrian dynamics or vehicle trajectories at intersections could cause incorrect speed adjustments, posing severe safety risks.

In intelligent traffic management and vehicle platooning, high AoI also matters. Stale traffic flow data may lead to mismatched signal cycles, while delayed platoon data could break synchronization and cause accidents. Traditional latency metrics cannot reflect this continuous information freshness requirement [15], making AoI a vital metric for reliability.

In response to the above challenges, Yan et al. [16] proposed a computation offloading framework based on deep reinforcement learning, which minimizes the weighted sum of AoI cost, energy consumption, and rental costs by jointly optimizing UAV trajectories, user associations, and offloading decisions. However, this approach has notable limitations: (1) it makes decisions based solely on the current state, lacking the ability to predict future network states, leading to decision lag in highly dynamic environments; (2) it overlooks the spatiotemporal correlation of vehicle movement trajectories, making it difficult to capture traffic patterns; and (3) it demonstrates insufficient adaptability to network environment changes, with room for improvement in learning efficiency and convergence.

This paper proposes a novel deep reinforcement learning framework based on LSTM trajectory prediction for resource allocation optimization in UAV-assisted VEC networks. The framework innovatively integrates vehicle trajectory prediction with the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm to enable proactive computation offloading and UAV trajectory planning.Contrasting with previous methods, the key contributions of this paper are presented as follows:

• A novel vehicle trajectory prediction network based on an LSTM attention mechanism is designed, which accurately forecasts the future positions and task requirements of vehicles. This provides a predictive decision-making foundation for resource optimization, effectively addressing the decision-making lag issue that traditional methods face in highly dynamic environments.
• State smoothing and data augmentation techniques are proposed to effectively mitigate the impact of environmental state fluctuations on learning stability, thereby enhancing the model’s robustness against noise and uncertainty.
• An innovative mechanism is developed to integrate prediction results into the optimization decision-making process, enabling the system to adjust current decisions based on predicted future states and achieve truly proactive resource optimization.

The remainder of this paper is structured as follows: Section 2 reviews the related literature; Section 3 details the system model; Section 4 proposes a TD3-based computation offloading optimization framework integrated with LSTM prediction; Section 5 presents the simulation results and performance analysis; and Section 6 concludes this paper with suggestions for future research.

2. Related Work

This section reviews the research progress related to UAV-assisted vehicular edge computing, encompassing vehicular edge computing systems, computing offloading strategies, the application of deep reinforcement learning in resource optimization, and trajectory prediction technologies. It aims to provide a theoretical background and research foundation for the innovative work presented in this paper.

2.1. Vehicular Edge Computing and UAV-Assisted Systems

VEC has emerged as a critical approach enabling autonomous driving-related technologies for intelligent transportation systems by deploying computing resources at the network edge to deliver low-latency, high-reliability computing services to vehicle users [17,18]. Liu et al. [19] explored the application scenarios and architectural design of VEC in the Internet of Vehicles, proposing a hierarchical VEC framework. Huang et al. [20] investigated the revenue and energy efficiency optimization in VEC networks, designing a deep reinforcement learning approach to delay-constrained task offloading and resource allocation. However, traditional statically deployed VEC systems still face challenges of insufficient coverage and uneven resource allocation in complex road environments.

In recent years, UAV-assisted mobile edge computing (UAV-MEC) has gained significant attention as an emerging technological paradigm [21]. Huda et al. [22] provided a comprehensive survey of UAV-MEC systems, analyzing their architectural design, resource management, and application scenarios. Wang et al. [23] and Wu et al. [24], respectively, investigated the impact of queue delay in UAV-MEC systems: the former established a statistically significant average delay model based on queuing theory, while the latter considered task arrival randomness, constructed a computing queue, and formalized it as a constrained Markov decision process (CMDP) problem.

In UAV-assisted VEC systems, the integration of software-defined networking (SDN) technology further enhances network management flexibility and programmability [25,26]. Luo et al. [27] proposed an SDN-driven collaborative data sharing framework to enable efficient data transmission in edge computing-assisted 5G-VANET. This paper leverages the SDN concept of separating control and data planes to construct a centralized global information collection and decision distribution mechanism, providing infrastructure support for proactive resource optimization.

2.2. Application of Deep Reinforcement Learning in Resource Optimization

Traditional optimization approaches, including game theory, convex optimization, and Lyapunov optimization, often struggle to handle highly dynamic and uncertain environments [28]. Deep reinforcement learning (DRL) has emerged as a powerful tool for addressing network resource optimization challenges, owing to its robust perception and decision-making capabilities [29].

In UAV-MEC systems, the application of DRL has made significant progress. Li et al. [30] proposed a multi-agent DRL method to solve the joint trajectory control and unloading allocation problem in multi-UAV-assisted MEC systems. Hao et al. [31] developed a multi-UAV collaborative edge computing system with joint task offloading with task priority to maximize the long-term average system gain. Pervez et al. [13] used DRL technology to jointly optimize communication and computing resources to reduce energy consumption and latency. Yan et al. [16] first applied the dual-delay deep deterministic policy gradient (TD3) algorithm to UAV-assisted VEC networks and achieved comprehensive optimization of system performance by considering the weighted cost of AoI, energy consumption, and rental price.

Although the above methods have achieved remarkable results in their respective research contexts, they all share a common limitation: decisions are based only on the existing state of environmental affairs and lack the ability to predict future states. In high-speed moving vehicle and UAV environments, this “passive response” decision-making strategy is often difficult to adapt to the rapidly changing network state, resulting in lag and non-optimality in resource allocation.

2.3. Sequence Prediction and Trajectory Prediction Technologies

Sequence prediction and trajectory prediction technologies have exhibited substantial potential in addressing time-series-related challenges. As a specialized variant of recurrent neural networks (RNNs), the long short-term memory network (LSTM) effectively overcomes the long-sequence dependency issue by incorporating a gating mechanism, demonstrating superior performance in time-series prediction tasks [32,33].

In traffic scenarios, trajectory prediction technology has been widely applied to vehicle behavior analysis and path planning [34]. Altché et al. [35] proposed an LSTM-based vehicle trajectory prediction model, which accurately forecasts vehicle movements in highway environments. M et al. [36] developed a vehicle trajectory prediction framework integrated with an attention mechanism, significantly enhancing prediction accuracy in complex traffic scenarios.

In recent years, the integration of prediction technology with decision-making approaches has emerged as an emerging research direction. Tu et al. [37] combined LSTM prediction with deep reinforcement learning, proposing a predictive resource allocation framework for mobile edge computing environments. Dong et al. [38] developed a prediction-driven deep reinforcement learning method to optimize edge server resource allocation decisions by forecasting user demand. However, in UAV-assisted VEC environments, research on the organic integration of trajectory prediction and resource optimization decision-making remains relatively scarce.

Most existing studies treat prediction and decision-making as independent components, lacking effective integration and feedback optimization mechanisms. Furthermore, the impact of prediction result uncertainty on decision-making quality has not been fully explored. The LSTM-based TD3 framework proposed in this paper not only achieves deep integration of prediction and decision-making but also enhances the system’s prediction accuracy and decision-making stability in highly dynamic environments through a prediction quality-aware training mechanism and state smoothing processing, offering novel insights for resource optimization in UAV-assisted VEC systems.

3. System Model

This section details the UAV-assisted vehicular edge computing system model incorporating LSTM prediction, encompassing the network architecture, communication model, computing model, energy consumption model, and information timeliness model. This provides the theoretical foundation for the subsequent optimization problems.

3.1. Network Architecture

Different from the software-defined network-driven UAV-assisted VEC architecture in [16], our control plane integrates an SDN controller, DRL agent, and LSTM predictor via cellular communication to enable collaborative processing of global information collection, future state estimation, and optimization decision-making. The data plane primarily employs dedicated short-range communication (DSRC) technology, comprising M RSUs and a rotorcraft UAV to provide edge computing services for N vehicles, as illustrated in Figure 1.

We define the set of roadside units (RSUs) as $\mathcal{M}=\{1,2,\dots ,m,\dots ,M\}$ and the set of vehicles as $\mathcal{N}=\{1,2,\dots ,n,\dots ,N\}$. For convenience, we combine UAVs and RSUs into a set of edge nodes (ENs) ${\mathcal{M}}^{\prime }=\{1,2,\dots ,m,\dots ,M\}$, where $m=M+1$ represents a UAV. Each edge node is equipped with a computing server and an SDN switch for communicating with the SDN controller.

In the control plane, the SDN controller periodically collects global data, including vehicle coordinates and task information, and synchronously forwards them to the LSTM predictor and DRL agent. The LSTM predictor generates future trajectory predictions of vehicles based on historical trajectory data, which are then transmitted to the DRL agent. The DRL agent integrates the prediction results with the current environmental state, outputs the optimization strategy via the TD3 algorithm, and achieves global coordinated control over UAV trajectory planning and task offloading decisions.

3.2. Communication Model

In the construction of the communication system model, vehicles can offload some computing tasks to edge nodes (ENs). We divide the finite time interval into T equal-length time slots, each with a length of $\tau$. For ease of analysis, the model uses a three-dimensional Cartesian coordinate system, the position of the roadside unit (RSU) is denoted as ${\mathbf{w}}_m=(x_m,y_m,0)$, the position of vehicle n at time slot t is denoted as $w_n\left(t\right)=\left(x_n\right(t),y_n(t),0)$, and the position of the UAV at time slot t is denoted as $w_u\left(t\right)=\left(x_u\right(t),y_u(t),H)$, where H is the fixed flight altitude of the UAV. The flight direction of the UAV is determined by the angle ${\theta }_u\left(t\right)\in [0,2\pi )$, and the flight speed is $v_u\left(t\right)\in [0,v_{m}ax)$. Therefore, the position update of the UAV can be expressed as follows:

$$x_u\left(t\right)=x_u(t-1)+\tau v_u\left(t\right)\cos \left({\theta }_u\right(t\left)\right),\forall t\in T$$

(1)

$$y_u\left(t\right)=y_u(t-1)+\tau v_u\left(t\right)\sin \left({\theta }_u\right(t\left)\right),\forall t\in T$$

(2)

Here, $\tau$ is the length of the time slot and T is the set of time slots.

For air-to-ground communication between UAV and vehicles, we adopt a probabilistic path loss model considering line-of-dight (LoS) and non-line-of-sight (NLoS) links [39]. The path loss between the UAV and vehicle n can be expressed as follows:

$$P{L}_{n,u}^{A2G}(t)=20\log \left({\displaystyle \frac{\left|\right|\mathbf{w}u\left(t\right)-\mathbf{w}n\left(t\right)\left|\right|}{4\pi f/C}}\right)+P_{n,u}^{LoS}(t)\eta LoS+(1-P_{n,u}^{LoS}(t\left)\right){\eta }_{NLoS}$$

(3)

where f is the carrier frequency, C is the speed of light, and ${\eta }_{LoS}$ and ${\eta }_{NLoS}$ are the average additional path losses for LoS and NLoS links, respectively. $P_{n,u}^{LoS}\left(t\right)=[1/(1+a\exp (-b\left({\theta }_{n,u}\right(t\left)\right)-a))]$ is the LoS link probability, a and b are environment parameters, and ${\theta }_{n,u}\left(t\right)=arcsin(H/||{\mathbf{w}}_u\left(t\right)-{\mathbf{w}}_n\left(t\right)\left|\right|)$ is the elevation angle between the UAV and vehicle n.

For ground-to-ground communication between the RSU and vehicles, we adopt the Rayleigh fading channel model, with path loss expressed as follows:

$$P{L}_{n,m}^{G2G}(t)=20\log \left({\displaystyle \frac{\left|\right|{\mathbf{w}}_m-\mathbf{w}n\left(t\right)\left|\right|}{4\pi f/C}}\right)+\eta Rayl$$

(4)

where ${\eta }_{Rayleigh}$ is the Rayleigh fading coefficient.

To avoid transmission interference, we adopt the Frequency-Division Multiple Access (FDMA) protocol. Define binary variable ${\alpha }_{n,m}(t)\in \{0,1\}$, representing the association state between vehicle n and edge node m at time slot t, where ${\alpha }_{n,m}\left(t\right)=1$ indicates that vehicle n is associated with edge node m and transmitting data; otherwise, ${\alpha }_{n,m}\left(t\right)=0$. Considering system resource limitations, we have the following constraints:

$$\sum _{m=1}^{M+1}{\alpha }_{n,m}(t)=1\forall n\in \mathcal{N},\forall t\in \mathcal{T}$$

(5)

$$\sum _{n\in \mathcal{N}}{\alpha }_{n,m}(t)\le N_M^{\max }\forall m\in \mathcal{M},\forall t\in \mathcal{T}$$

(6)

where $N_M^{\max }$ represents the maximum number of vehicles each edge node can serve.

Additionally, we define ${\beta }_{n,m}\left(t\right)\in [0,1]$ as the bandwidth resource proportion allocated by edge node m to vehicle n. For simplicity, assume ${\beta }_{n,m}\left(t\right)=1/\left({\sum }_{n\in \mathcal{N}}{\alpha }_{n,m}\right(t\left)\right)$; i.e., all vehicles associated with the same edge node equally share bandwidth resources.

According to Shannon’s theorem, the transmission rate between vehicle n and edge node m at time slot t is

$$R_{n,m}(t)={\beta }_{n,m}(t)W_M{\log }_2\left(1+{\displaystyle \frac{p_n\left(t\right)h_{n,m}\left(t\right)}{{\sigma }_{n,m}^2\left(t\right)}}\right)$$

(7)

where $W_M=W_R$ (or $W_U$) is the total uplink bandwidth of each RSU (or UAV), $p_n\left(t\right)$ is the transmission power of vehicle n, ${\sigma }_{n,m}^2\left(t\right)={\beta }_{n,m}\left(t\right)N_0{W}_M$ is the average power of Additive White Gaussian Noise (AWGN), $N_0$ is the AWGN spectral density, and $h_{n,m}\left(t\right)={10}^{(-P{L}_{n,m}(t)/10)}$ is the channel gain, where $P{L}_{n,m}\left(t\right)=P{L}^{G2G}n,m\left(t\right)$ (when $m\in \mathcal{M}$) or $P_{n,m}\left(t\right)=P{L}_{n,m}^{A2G}\left(t\right)$ (when $m=M+1$).

3.3. Computing Model

In this system, we adopt a partial offloading strategy, allowing vehicles to allocate tasks in arbitrary proportions among local, RSU, or UAV processing. At time slot t, vehicle n randomly generates computational task $U_n\left(t\right)$, with the task arrival rate following a Poisson distribution with parameter ${\lambda }_u$. Specifically, $U_n\left(t\right)=\left(D_n\right(t),T_n^{\max }(t\left)\right)$, where $D_n\left(t\right)$ (unit: bits) represents input data size and $T_n^{\max }\left(t\right)$ (unit: seconds) represents the task’s maximum tolerable delay. Define $o_n\left(t\right)\in [0,1]$ as the computational offloading proportion of vehicle n at time slot t; i.e., vehicle n offloads an $o_n\left(t\right)$ portion of the task to the associated edge node m for remote computation, with the remaining $(1-o_n(t\left)\right)$ portion executed locally. To cache local computation tasks and edge node offloading tasks, we define task computation queues $Q_n\left(t\right)$ and $Q_m\left(t\right)$ for vehicle $n\in \mathcal{N}$ and edge node $m\in {\mathcal{M}}^{\prime }$, respectively. The evolution of these queues across time slots is as follows:

$$Q_n(t+1)=\max \left\{Q_n\right(t)-D_n^{\mathrm{comp}}(t),0\}+(1-o_n(t\left)\right)D_n\left(t\right)$$

(8)

$$Q_m(t+1)=\max \left\{Q_m\right(t)-D_m^{\mathrm{comp}}(t),0\}+\sum _{n\in \mathcal{N}}{\alpha }_{n,m}(t)o_n(t)D_n(t)$$

(9)

where $D_n^{\mathrm{comp}}\left(t\right)=\min \{\tau F_n/X_c,Q_n(t\left)\right\}$ and $D_m^{\mathrm{comp}}\left(t\right)=min\tau F_m/X_c,Q_m\left(t\right)$, respectively, represent the amount of data (unit: bits) that vehicle n and edge node m can process at time slot t. $F_n$ represents the local computing capability of vehicle n (unit: CPU cycles/second), $F_m=F_R$ (or $F_U$) represents the computing capability of the RSU (or UAV) (unit: CPU cycles/second), and $X_c$ represents the number of CPU cycles required to process 1 bit of data.

Based on the above model [40], we calculate service delay and energy consumption for local computation and computational offloading, respectively.

• Local Computation: For the local computation proportion $(1-o_n(t\left)\right)$ of task $U_n\left(t\right)$, local service delay includes queue waiting time and computation time:

  $${\tau }_n^{\mathrm{local}}\left(t\right)={\tau }_n^{\mathrm{queue}}\left(t\right)+{\tau }_n^{\mathrm{comp}}\left(t\right)$$

  (10)

  where

  $${\tau }_n^{\mathrm{queue}}(t)={\displaystyle \frac{X_c{Q}_n\left(t\right)}{F_n}}$$

  (11)

  $${\tau }_n^{\mathrm{comp}}(t)={\displaystyle \frac{(1-o_n\left(t\right))X_c{D}_n\left(t\right)}{F_n}}$$

  (12)
  In accordance with the commonly adopted power consumption model $k_n{\left(F_n\right)}^{v_n}$, where $k_n\ge 0$ is the effective capacitance coefficient related to the processor chip architecture and $v_n$ is typically set to 3, the local computation energy consumption of vehicle n at time slot t is

  $$E_n^{\mathrm{local}}\left(t\right)=k_n{\left(F_n\right)}^3{\tau }_n^{\mathrm{comp}}\left(t\right)=k_n{\left(F_n\right)}^2(1-o_n(t\left)\right)X_c{D}_n\left(t\right)$$

  (13)
• Computational Offloading: If ${\alpha }_{n,m}\left(t\right)=1,m\in {\mathcal{M}}^{\prime }$, then the $o_n\left(t\right)$ portion of task $U_n\left(t\right)$ will be offloaded to edge node m. The offloading service delay is the sum of transmission time, queue waiting time, and computation time:

  $${\tau }_n^{\mathrm{off}}\left(t\right)={\tau }_{n,m}^{\mathrm{trans}}\left(t\right)+{\tau }_{n,m}^{\mathrm{queue}}\left(t\right)+{\tau }_{n,m}^{\mathrm{comp}}\left(t\right)$$

  (14)

  where

  $${\tau }_{n,m}^{\mathrm{trans}}(t)={\displaystyle \frac{o_n\left(t\right)D_n\left(t\right)}{Rn,m\left(t\right)}}$$

  (15)

  $${\tau }_{n,m}^{\mathrm{queue}}(t)={\displaystyle \frac{X_c{Q}_m\left(t\right)}{F_m}}$$

  (16)

  $${\tau }_{n,m}^{\mathrm{comp}}(t)={\displaystyle \frac{o_n\left(t\right)X_c{D}_n\left(t\right)}{F_m}}$$

  (17)
  Since the data volume of computation results returned by edge nodes is typically much smaller than that of the original task data, we ignore the result return time. Correspondingly, the offloading energy consumption of vehicle n is mainly transmission energy consumption:

  $$E_{n,m}^{\mathrm{off}}\left(t\right)=p_n\left(t\right){\tau }_{n,m}^{\mathrm{trans}}\left(t\right)$$

  (18)

In summary the total energy consumption required for the computing task is the sum of the local computing energy consumption and the computing offloading energy consumption, i.e.,

$$E_n^{total}=E_n^{local}(t)+{\alpha }_{n,m}(t)E_{n,m}^{off}(t)$$

(19)

3.4. UAV Energy Consumption Model

• UAV Computation Energy Consumption: Similarly to the vehicle computing energy consumption model, the computing energy consumption of UAV in time slot t is

  $$E_u^{\mathrm{comp}}\left(t\right)=k_u{\left(F_U\right)}^3{\tau }_{n,m}^{\mathrm{comp}}\left(t\right),m=M+1$$

  (20)
• Propulsion Energy Consumption: According to the study [41], the flight energy consumption of rotary-wing UAVs is related to their flight speed, expressed as follows:

  $$\begin{array}{cc}\hfill E_u^{\mathrm{propul}}(t)& =\tau [P_0\left(1+{\displaystyle \frac{3\left|\right|v_u\left(t\right){\left|\right|}^2}{U_{\mathrm{tip}}^2}}\right)\hfill \\ \hfill & +P_i\sqrt{1+{\displaystyle \frac{\left|\right|v_u\left(t\right){\left|\right|}^4}{4v_0^4}}-{\displaystyle \frac{\left|\right|v_u\left(t\right){\left|\right|}^2}{2v_0^2}}}\hfill \\ \hfill & +{\displaystyle \frac{1}{2}}{d}_0{\rho }_0{s}_0{A}_0\left|\right|v_u(t){\left|\right|}^3]\hfill \end{array}$$

  (21)

  where $v_u\left(t\right)=\left|\right|{\mathbf{w}}_u(t+1)-\mathbf{w}u\left(t\right)\left|\right|/\tau$ represents UAV speed, $P_0$ and $P_i$ are the blade profile power and induced power in hovering state, respectively, $Utip$ is rotor blade tip speed, $v_0$ is average rotor-induced speed in hovering state, $d_0$ is the fuselage drag ratio, ${\rho }_0$ is air density, $s_0$ is rotor solidity, and $A_0$ is rotor disc area. When the UAV hovers (i.e., flight speed is 0), the hovering energy consumption is $E_u^h=\tau (P_0+P_i)$.

In summary, the total energy consumption of UAV at time slot t is

$$E_u(t)=E_u^{\mathrm{comp}}(t)+E_u^{\mathrm{propul}}(t)$$

(22)

3.5. AoI Model

AoI is a destination node-centric metric used to measure information freshness in networks [42]. In this system, we define the AoI of vehicle n at time slot t as ${\Delta }_n\left(t\right)$, representing the time difference between current time t and the generation time of the most recent computation result received by vehicle n:

$${\Delta }_n\left(t\right)=t-g_n\left(t\right)$$

(23)

where $g_n\left(t\right)$ represents the generation time of the most recent computation result received by vehicle n before time slot t. As shown in Figure 2, for vehicle n under the FCFS principle, its AoI accumulates linearly over time until a task computation completes and returns results, at which point AoI drops to the processing delay value of that task. We define the generation time of task u as $t_{n,u}$, computation completion time as $t_{n,u}^{\prime }$, execution time as $T_{n,u}=t_{n,u}^{\prime }-t_{n,u}$, and inter-task generation interval as $Y_{n,u}=t_{n,u+1}-t_{n,u}$.

According to Figure 2, the AoI update rule for vehicle n is

$${\Delta }_n(t+1)=\left\{\begin{array}{cc}{T}_{n,u}+1,\hfill & \mathrm{if}t=t_{n,u}^{\prime }\hfill \\ {\Delta }_n\left(t\right)+1,\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(24)

We use Peak AoI (PAoI) as the performance metric [43], i.e., the maximum AoI value before the vehicle receives computation results. The PAoI for task u is expressed as follows:

$$A_{n,u}=\left\{\begin{array}{cc}{\Delta }_n\left(0\right)+T_{n,1},\hfill & \mathrm{if}u=1\hfill \\ Y_{n,u-1}+T_{n,u},\hfill & \mathrm{if}u>1\hfill \end{array}\right.$$

(25)

For the partial offloading scheme, the total execution delay of task u is the maximum of local computation and offloading computation delays:

$$T_{n,u}=\max \{{\tau }_{n,u}^{\mathrm{local}},{\tau }_{n,u}^{\mathrm{off}}\}$$

(26)

Assuming ${\Delta }_n\left(0\right)=0$, the PAoI of vehicle n can be expressed as follows:

$${\Delta }_n^{\mathrm{total}}\left(t\right)=\max \left\{{\tau }_n^{\mathrm{local}}\right(t),{\tau }_n^{\mathrm{off}}(t\left)\right\}+Y_n\left(t\right)$$

(27)

where $Y_n\left(t\right)$ represents the task generation interval of vehicle n. Based on the above system model, we will construct an LSTM prediction-based TD3 optimization framework in the next section, aiming to achieve joint optimization of AoI, energy consumption, and resource rental costs by predicting future vehicle trajectories and task demands.

3.6. Total Cost Function

The total system cost is determined by the AoI cost, energy consumption cost, and computing resource rental cost. We also need to consider the rental price of EN servers. The unit price of computing resources rented by the RSU and UAV (in pennies per CPU cycle) is expressed as $P_m=P_R,m\in M$ or $P_m=P_U,m=M+1$. Therefore, the total price paid by vehicle n to EN m is

$$P_n^{total}\left(t\right)=P_m{o}_n\left(t\right)X_c{D}_n\left(t\right)$$

(28)

All factors affecting the total cost of the system have been calculated, and the total cost can be expressed as

$$C_n^{total}={\gamma }_A{A}_n^{total}\left(t\right)+{\gamma }_E{E}_n^{total}\left(t\right)+{\gamma }_P{P}_n^{total}\left(t\right)$$

(29)

where ${\gamma }_A$, ${\gamma }_E$, and ${\gamma }_P$ represent the weight coefficients of AoI cost, energy consumption, and rental price, respectively, and ${\gamma }_A+{\gamma }_E+{\gamma }_P=1$.

Our goal is to optimize UAV trajectories, user associations, and offloading assignments to minimize the long-term costs of N vehicles. We define two sets $A\left(t\right)={\left\{{\alpha }_{n,m}\right(t\left)\right\}}_{n\in N,m\in M}$ and $O\left(t\right)={\left\{o_n\right(t\left)\right\}}_{n\in N}$. Then, the total cost minimization problem can be formulated as

$$\begin{array}{cc}\hfill \mathrm{OP}1:& \underset{{\mathbf{w}}_n\left(t\right),\mathbf{A}\left(t\right),\mathbf{O}\left(t\right)}{min}\sum _{t\in \mathcal{T}}\sum _{n\in \mathcal{N}}{C}_n^{\mathrm{total}}(t)\hfill \end{array}$$

(30)

Since $A\left(t\right)$ is a set of discrete variables and there is a non-functional $\max \{·\}$ in the objective function, it is difficult to obtain the optimal solution of Formula (30) using traditional methods. Therefore, we choose the DRL method to solve this problem.

4. LSTM Prediction-Based TD3 Computing Offloading Optimization Framework

This section introduces in detail our proposed TD3 computation offloading optimization framework based on LSTM trajectory prediction, which achieves forward-looking resource optimization by predicting the future trajectory of the vehicle and effectively solves the decision lag problem in the UAV-assisted VEC network.

4.1. MDP Formulation

In this network model, the dynamic evolution of vehicle and edge node (EN) buffer queues, the continuity of UAV flight trajectories, the time-varying channel gain, the real-time vehicle positions, and the execution of computing tasks in each time slot all exhibit Markovian properties. Meanwhile, the environment’s constant dynamics make it challenging to derive state transition probabilities, thus justifying the use of a model-free MDP [44]. The model-free MDP usually consists of a 3-tuple $(S,A,R)$, where S represents the system state space, A represents the action space, and R represents the reward function.

Specifically, the operation process of the system is as follows: At any time slot t, the SDN controller collects global information and sends it to the DRL agent, and the current environment state is represented as $s_t\in S$. The agent selects an action from A according to the strategy $\pi \left(a_t\right|s_t)$, interacts with the environment, and immediately obtains a reward of $r_t(s_t,a_t)\in R$. Then, the environment state transitions to the next state $s_{t+1}\in S$. Based on the rewards obtained, the agent continuously improves its strategy and iteratively learns through continuous interaction with the environment until the optimal strategy is obtained.

4.1.1. State Space S

At the beginning of each time slot, the SDN controller captures the key parameters and global status information of the network environment in real time and transmits these data to the intelligent agent after integration. At this time, the time slot t can be defined as follows:

$$s_t=\left\{W\right(t),U(t),Y(t),Q(t),H(t),E_U^{res}(t\left)\right\}$$

(31)

where $W\left(t\right)$, $U\left(t\right)$, $Y\left(t\right)$, $Q\left(t\right)$, $H\left(t\right)$, and $E_U^{res}\left(t\right)$ are defined as follows.

• $W\left(t\right)$ is the set of position coordinates of all vehicles and UAVs at time slot t. The heights of vehicles and UAVs remain unchanged, so only their horizontal coordinates need to be considered. The vector dimension is $2(N+1)$.
• $u\left(t\right)$ represents the task information of all vehicles at time slot t, including the current task size of each vehicle and the maximum tolerable delay in completing the task. The vector dimension is $2N$.
• $Y\left(t\right)$ represents the task generation interval of all vehicles at time slot t, reflecting the time since each vehicle last generated a task. The vector dimension is N.
• $Q\left(t\right)$ represents the queue backlog status at time slot t, including the local queues, RSU queues, and UAV queues of all vehicles. The vector dimension is $N+M+1$.
• $H\left(t\right)$ describes the channel state information at time slot t, including the channel gain between the vehicle and the UAV and RSU. The vector dimension is $N(M+1)$.
• $E_U^{res}\left(t\right)$ represents the remaining energy of the UAV at time slot t. Its update follows $E_u^{res}(t+1)=\max \left\{E_u^{res}\right(t)-E_u(t),0\},E_u^{res}\left(0\right)=E_u^{max}$

4.1.2. Action Space A

The action space consists of two parts: UAV trajectory control and vehicle unloading decision:

• UAV trajectory $w_u\left(t\right)$: As mentioned earlier, the trajectory control of the UAV is determined by the speed and heading angle, i.e., $w_u\left(t\right)=\left\{v_u\right(t),{\theta }_u(t\left)\right\}$.
• User Association $A\left(t\right)$: The user-related variables ${\alpha }_{n,m}\left(t\right)$ required for calculation in the system model are discrete variables and cannot be directly output from the action space. Therefore, we designed a mapping method that outputs continuous variables ${\alpha }_n\left(t\right)$ in the action space and maps them to ${\alpha }_{n,m}\left(t\right)$ through Equation (32).

  $${\alpha }_{n,m}(t)=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}{\mathbf{1}}_{\{d_{n,m}\le L/M\}}{\alpha }_n\left(t\right)\in \left({\displaystyle \frac{1}{3}},{\displaystyle \frac{2}{3}}\right],m\in \mathcal{M},\hfill \\ \hfill & or{\alpha }_n\left(t\right)\in \left({\displaystyle \frac{2}{3}},1\right),m=M+1\hfill \\ 0,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

  (32)
  Here, the indicator function ${\mathbf{1}}_{\{·\}}$ takes the value of 1 when the condition $\{·\}$ is met; otherwise it is 0. The introduction of the transition variable ${\alpha }_n\left(t\right)$ not only ensures the continuity of the action space of the deep reinforcement learning algorithm, but also satisfies the constraint that each vehicle can only be associated with one EN in a single time slot. Specifically, the following hold:
  • When ${\alpha }_n\left(t\right)$ falls within the interval $\left(0,{\displaystyle \frac{1}{3}}\right]$, it means that the vehicle chooses to perform the computing task locally, and all values of ${\alpha }_{n,m}\left(t\right)$,$m\in M^{\prime }$ are 0;
  • When ${\alpha }_n\left(t\right)$ is in the interval $\left({\displaystyle \frac{1}{3}},{\displaystyle \frac{2}{3}}\right]$, it means that the vehicle decides to offload the task to the RSU m on its road for processing, corresponding to ${\alpha }_{n,m}\left(t\right)=1$ and $m\in M$;
  • When ${\alpha }_n\left(t\right)$ is in the interval $\left({\displaystyle \frac{2}{3}},1\right]$, it means that the vehicle chooses to offload the task to the UAV for processing, and the corresponding ${\alpha }_{n,m}\left(t\right)=1$ and $m=M+1$.
• Task Offloading Decision $O_n\left(t\right)$: To reduce the dimension of the action space, the task offloading decision $O_n\left(t\right)$ is mapped and coupled with the user association decision. Given that the offloading allocation variable ${\alpha }_n\left(t\right)$ is divided into three intervals, by constructing the mapping relationship $o_n\left(t\right)=3{\alpha }_n\left(t\right)-\lfloor 3{\alpha }_n\left(t\right)\rfloor$, the offloading ratio $O_n\left(t\right)$ can be continuously taken in the interval [0,1]. We define

  $$o_n(t)=\left\{\begin{array}{cc}3{\alpha }_n\left(t\right)-\lfloor 3{\alpha }_n\left(t\right)\rfloor ,\hfill & \mathrm{if}{\alpha }_n\left(t\right)\in \left({\displaystyle \frac{1}{3}},{\displaystyle \frac{2}{3}}\right)\cup \left({\displaystyle \frac{2}{3}},1\right)\hfill \\ 1,\hfill & \mathrm{if}{\alpha }_n\left(t\right)\in \{{\displaystyle \frac{2}{3}},1\}\hfill \\ 0,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

  (33)
  Therefore, the action space only contains the trajectory control variables of the UAV and the user-related variables used to determine the unloading destination and unloading ratio, which are expressed as

  $$a_t=\left\{V_u\right(t),{\theta }_u(t),{\alpha }_1(t),···,{\alpha }_n(t),···,{\alpha }_N(t\left)\right\}$$

  (34)

User association mapping in Formula (32): This design aims to solve the adaptation problem of the DRL continuous action space and discrete association constraints. By mapping the continuous variable ${\alpha }_n\left(t\right)$ to the “local/RSU/UAV” association state in segments, the continuity of the action space is guaranteed (meeting the requirements of the TD3 algorithm), and the unique user–node association is naturally realized through the constraint of $\sum {\alpha }_{n,m}\left(t\right)=1$.

4.1.3. Reward Function R

With the aim of minimizing the total system cost, we define a reward function that is negatively associated with the cost function and add three penalty terms, defined as

$$r_t=-\sum _{n\in \mathcal{N}}{C}_n^{\mathrm{total}}(t)-{\varphi }_1(t)-\sum _{n\in \mathcal{N}}{\varphi }_{2n}(t)-{\varphi }_3(t)$$

(35)

where

$$\begin{array}{cc}\hfill {\varphi }_1\left(t\right)& =\left\{\begin{array}{cc}{c}_1,\hfill & \mathrm{if}\sum _{n\in \mathcal{N}}{\alpha }_{n,m}\left(t\right)>N_M^{\max }\mathrm{for}\mathrm{any}m\in \mathcal{M}\hfill \\ & \mathrm{or}\sum _{n\in \mathcal{N}}{\alpha }_{n,m}\left(t\right)>N_U^{\max }\mathrm{for}m=M+1\hfill \\ 0,\hfill & \mathrm{otherwise}\hfill \end{array}\right.\hfill \end{array}$$

(36)

$$\begin{array}{cc}\hfill {\varphi }_{2n}\left(t\right)& =\left\{\begin{array}{cc}{c}_2,\hfill & \mathrm{if}\max \left\{{\tau }_n^{\mathrm{local}}\left(t\right),{\tau }_n^{\mathrm{off}}\left(t\right)\right\}\ge T_n^{\max }\left(t\right)\hfill \\ 0,\hfill & \mathrm{otherwise}\hfill \end{array}\right.\hfill \end{array}$$

(37)

$$\begin{array}{cc}\hfill {\varphi }_3\left(t\right)& =\left\{\begin{array}{cc}{c}_3,\hfill & \mathrm{if}{E}_u^{\mathrm{res}}\left(t\right)<0\hfill \\ 0,\hfill & \mathrm{otherwise}\hfill \end{array}\right.\hfill \end{array}$$

(38)

where $c_1$, $c_2$, $c_3$ > 0 is a constant that defines the penalty amount. ${\varphi }_1\left(t\right)$ is the user association constraint penalty, which is the penalty when the vehicle exceeds all service ranges; ${\varphi }_2\left(t\right)$ is the delay constraint penalty, which is the penalty when the delay in task processing exceeds the maximum acceptable limit; ${\varphi }_3\left(t\right)$ is the UAV energy constraint penalty, which is the penalty when the UAV has insufficient energy.

The objective of an MDP is to derive an optimal policy $\pi :S\to A$ for maximizing the expected long-term discounted cumulative reward, i.e.,

$$\underset{\pi }{\max }\underset{k\to \infty }{lim}{\mathbb{E}}_{\pi }\left[\sum _{t=0}^k{\zeta }^t{r}_t\right]$$

(39)

where $E_{\pi }(·)$ represents the expected value after following the strategy $\pi$. The discount factor $\zeta \in [0,1]$ is used to measure the impact of future rewards on the current state. When $\zeta$ is close to 0, the agent focuses on immediate benefits and ignores possible future rewards; when $\zeta$ is close to 1, the agent will regard long-term benefits and short-term benefits as equally important, and fully consider the long-term impact of decisions.

Therefore, we can solve the problem in Formula (30) using the algorithm based on DRL we proposed. OP1 can be transformed into

$$\begin{array}{cc}\hfill & \mathrm{OP}2:\underset{v_n\left(t\right),a_n\left(t\right),n\in \mathbb{N}}{\max }\left({\displaystyle \frac{1}{T}}\sum _{t=1}^T{r}_t\right)\hfill \\ \hfill & \mathrm{s}.\mathrm{t}.\hfill \end{array}$$

(40)

$$\begin{array}{cc}\hfill & v_n\left(t\right)\in \left[0,v^{\max }\right]\hfill \end{array}$$

(40a)

$$\begin{array}{cc}\hfill & a_n\left(t\right)\in \left[0,1\right],n\in \mathbb{N}\hfill \end{array}$$

(40b)

4.2. LSTM-TD3

This section details the TD3-based computation offloading optimization framework we proposed, which integrates LSTM trajectory prediction. The framework achieves proactive resource optimization by forecasting vehicle future trajectories, effectively addressing the decision-making lag issue in UAV-assisted VEC networks.

As depicted in Figure 3, the proposed LSTM-TD3 framework consists of four core components: a vehicle trajectory predictor, state processor, TD3 optimizer, and decision fusion mechanism. In contrast to traditional reinforcement learning methods that rely solely on the current state for decision-making, our framework realizes prediction-driven decision optimization through a prediction–decision dual closed-loop mechanism, alongside spatiotemporal correlation perception, proactive state optimization, and prediction quality adaptation.

4.2.1. Vehicle Trajectory Predictor

The trajectory prediction network we designed adopts an encoder–decoder architecture to improve sequence modeling capabilities. The network can capture both short-term and long-term dependencies of vehicle trajectories. The prediction network mainly consists of a feature encoding layer, a multi-layer LSTM encoder, a self-attention mechanism, and a prediction decoder. For the input sequence $\mathbf{X}=[{\mathbf{x}}_1,{\mathbf{x}}_2,...,{\mathbf{x}}_T]$, where ${\mathbf{x}}_t\in {\mathbb{R}}^d$ is the d-dimensional eigenvector at time t.

The prediction process can be described as follows: First, the input sequence is normalized and dimensionally transformed:

$${\tilde{\mathbf{x}}}_t=\mathrm{LayerNorm}\left({\mathbf{x}}_t\right)$$

(41)

$${\mathbf{e}}_t=\mathrm{ReLU}({\mathbf{W}}_e{\tilde{\mathbf{x}}}_t+{\mathbf{b}}_e)$$

(42)

Then the multi-layer LSTM processes the encoded sequence:

$${\mathbf{h}}_t^l,{\mathbf{c}}_t^l={\mathrm{LSTM}}^l({\mathbf{h}}_t^{l-1},{\mathbf{h}}_{t-1}^l,{\mathbf{c}}_{t-1}^l)$$

(43)

Then the self-attention mechanism is applied to capture the long-distance dependencies within the sequence; finally, the LSTM output and the attention context vector are combined for prediction:

$$\mathbf{z}=[{\mathbf{h}}_T^L;{\mathbf{c}}_T]$$

(44)

$$\widehat{\mathbf{y}}={\mathbf{W}}_o\mathrm{ReLU}({\mathbf{W}}_z\mathbf{z}+{\mathbf{b}}_z)+{\mathbf{b}}_o$$

(45)

To fully capture the kinematics and task characteristics of the vehicle, we extract a six-dimensional feature vector for each vehicle:

$${\mathbf{f}}_i={[x_i,y_i,v_i,{\theta }_i,D_i,{\lambda }_i]}^T$$

(46)

where $(x_i,y_i)$ represents the position coordinates of the vehicle in the two-dimensional plane, $v_i$ is the vehicle speed, ${\theta }_i$ is the driving direction, $D_i$ is the current task data size, and ${\lambda }_i$ is the task generation rate.

4.2.2. State Processor

The core innovation of the LSTM-TD3 framework is to fuse the predicted future state with the current observed state to achieve forward-looking decision-making. The state processor is responsible for implementing the following fusion process:

$$s_t^{*}=\mathcal{F}({\tilde{s}}_t,{\widehat{s}}_{t+k},c^t)$$

(47)

where ${\tilde{s}}_t$ is the current state after smoothing, ${\widehat{s}}_{t+k}$ is the state at time $t+1$ predicted based on the information at time t, $c^t$ is the prediction confidence, and $\mathcal{F}(·)$ is the fusion function. The fusion function we designed adopts the weighted average form

$$s_t^{*}=(1-c^t){\tilde{s}}_t+c^t{\widehat{s}}_{t+k}$$

(48)

This formula dynamically adjusts the weights of the current state and the predicted state according to the prediction confidence $c^t$. When the prediction quality is high, the system is more inclined to trust the predicted state and achieve forward-looking decision-making; when the prediction quality is low, the system relies more on the current observation state to maintain robustness.

State fusion dynamically balances the weights of the current state and the predicted state via prediction confidence $c^t$, with its core focus on addressing the question of “how prediction errors influence state inputs”. When prediction errors are high, the weight assigned to the predicted state is automatically reduced to mitigate its interference with decision-making. This ensures more robust foundational information for subsequent optimization processes and avoids decision biases that might arise from over-reliance on either predictions alone or the current state in isolation.

4.2.3. TD3 Optimizer

The TD3 algorithm is an advanced deep reinforcement learning method designed for continuous action spaces. The target value calculation formula for TD3 is as follows:

$$y=r+\gamma \underset{i=1,2}{min}{Q}_{{\theta }^{\prime }i}(s^{\prime },\pi {\varphi }^{\prime }\left(s^{\prime }\right)+ϵ)$$

(49)

where $ϵ\sim \mathrm{clip}\left(\mathcal{N}\right(0,\sigma ),-c,c)$ is the truncated Gaussian noise and ${\theta }^{\prime }i$ and ${\varphi }^{\prime }$ are the parameters of the target critic network and the target actor network, respectively.

Different from the standard TD3, we further introduce the prediction enhancement feature and modify the target value calculation:

$$y=r+\gamma \underset{i=1,2}{min}{Q}_{{\theta }^{\prime }i}(s_{t+1}^{*},{\pi }_{{\varphi }^{\prime }}\left(s_{t+1}\right)+ϵ)$$

(50)

where $s_{t+1}^{*}$ is the fused state generated by the state processor, which contains prediction information. This modification enables the optimizer to consider future state changes and achieve forward-looking decision-making.

4.2.4. Decision Fusion

The decision fusion mechanism is a key innovative component of the LSTM-TD3 framework, which organically integrates trajectory prediction, state processing, and optimization decision-making to achieve prediction-driven adaptive resource allocation.

Mathematically, the entire fusion process can be expressed as follows:

$$a_t^{*}=\mathcal{G}(s_t,{\widehat{s}}_{t+k},\pi \left(s_t\right),\pi \left(s_t^{*}\right),c^t,\mathcal{K}t)$$

(51)

where $s_t$ is the current observed state, ${\widehat{s}}_{t+k}$ is the predicted state for the next k moments, $\pi \left(s_t\right)$ is the policy output based on the current state, $\pi \left(s_t^{*}\right)$ is the policy output based on the fusion state, $c^t$ is the prediction confidence, ${\mathcal{K}}_t$ is the historical knowledge base, and $\mathcal{G}(·)$ is the decision fusion function.

The core of the decision fusion mechanism is to dynamically adjust the weights of the current decision and the prediction-guided decision according to the prediction quality. We design an adaptive weighted fusion model:

$$a_t^{*}=(1-{\omega }_t)·\pi \left(s_t\right)+{\omega }_t·\pi \left(s_t^{*}\right)$$

(52)

where ${\omega }_t$ is the fusion weight, which is determined by the prediction confidence $c^t$ and the environment dynamics $d^t$:

$${\omega }_t=c^t·\Phi \left(d^t\right)$$

(53)

$\Phi \left(d^t\right)$ is the modulation function of the environment dynamics:

$$\Phi (d^t)={\displaystyle \frac{1}{1+\exp (-\kappa ·(d^t-d_0))}}$$

(54)

$$d^t={\displaystyle \frac{1}{N}}\sum _n\parallel v_n(t)-v_n(t-1)\parallel +{\displaystyle \frac{1}{M+1}}\sum _m\parallel Q_m\left(t\right)-Q_m(t-1)\parallel /Q_m(t-1)$$

(55)

$d_0$ is the threshold parameter, and $\kappa$ controls the function steepness. When the environment is highly dynamic, $\Phi \left(d^t\right)$ is close to 1, and the system is more inclined to believe in prediction-guided decisions; when the environment is relatively static, $\Phi \left(d^t\right)$ is close to 0, and the system is more inclined to the current optimal decision. The dynamics of the environment $d^t$ is quantified by indicators such as vehicle speed, queue change rate, and channel state change rate. The prediction confidence $c^t$ is calculated by evaluating the historical prediction error:

$$c^t=\exp \left(-\lambda ·{\displaystyle \frac{1}{w}}\sum _{i=t-w}^{t-1}{e}_i\right)$$

(56)

where $e_i$ is the historical prediction error, $\lambda$ is the scaling factor, and w is the window size.

Decision fusion is based on prediction confidence and environmental dynamics $d^t$ to adjust weights ${\omega }_t$, focusing on solving the problem of “how to adapt to the dynamic needs of the scene”. High-dynamic scenes increase the weight of prediction-oriented decisions to respond in advance, while low-dynamic scenes focus on current state decisions to reduce redundant adjustments, achieving accurate adaptation of decision strategies in different scenarios.

4.2.5. Framework Process

At the beginning of each episode, the system randomly initializes the environment, including the initial positions of vehicles and UAVs, the backlog status of task queues for vehicles and edge nodes (ENs), the remaining energy of UAVs, and wireless channel conditions. Subsequently, the core components of the algorithm are initialized to construct a complete prediction and decision framework. Meanwhile, an experience replay buffer is developed to retain experience triples ($s_t$, $a_t$, r, $s_{t+1}$) of the agent’s interaction with the environment; a vehicle data buffer is created to store the time series of vehicle motion and task characteristics $f_i$.

At each time step t, the system first collects the global state information st through the SDN controller, including vehicle location, task information, queue status, and channel conditions. The vehicle trajectory predictor extracts the historical sequence $f_i$ from the vehicle data buffer and uses a multi-layer LSTM encoder and self-attention mechanism to predict the vehicle state $s_{t+k}$ in the next k steps.

As the core fusion component, the state processor dynamically calculates the fusion weight based on the prediction confidence $c^t$ and performs weighted fusion on the current observed state st and the predicted state $S_{t+k}$ to generate the enhanced state $s_t^{*}=(1-c^t){\tilde{s}}_t+c^t{s}_{t+k}$, where ${\tilde{s}}_t$ is the current state after smoothing.

The TD3 optimizer generates an action at based on the enhanced state $s_t^{*}$ through the actor network, including UAV trajectory control and task offloading decisions. After the system executes the action, it obtains the reward rt and the next state st+1. The experience quadruple $(s_t,a_t,r_t,s_{t+1})$ is stored in the playback buffer for TD3 network training, and the vehicle feature data is stored in a dedicated buffer for LSTM prediction network learning.

As the final decision-making component, the decision fusion device calculates the adaptive fusion weight ${\omega }_t$ according to the prediction confidence $c^t$ and the environment dynamics $d_t$, performs weighted fusion of the current optimal strategy $\pi \left(s_t^{*}\right)$ and the forward-looking strategy $\pi \left(s_{t+k}\right)$, and outputs the final execution action $a_t=(1-{\omega }_t)\pi \left(s_t^{*}\right)+{\omega }_t\pi \left(s_{t+k}\right)$. This two-layer fusion mechanism not only ensures the effective use of prediction information but also ensures the robustness of decision-making through adaptive weight adjustment, realizing true prediction-driven adaptive resource allocation. Algorithm 1 shows the detailed workflow of LSTM-TD3.    

Algorithm 1: LSTM prediction-based TD3 computation offloading optimization

5. Experiments

In this section, we substantiate the effectiveness of the LSTM-TD3 framework in UAV-assisted VEC network resource optimization through extensive simulation experiments. We first introduce the simulation environment setup and benchmark method and then analyze the performance advantages of our method in detail from multiple aspects.

5.1. Simulation Parameter Settings

To validate the proposed method, we constructed an end-to-end simulation platform using Python 3.12.1 and PyTorch 2.6.0 and conducted simulation experiments on a computer equipped with an Intel i7-14650K CPU(Intel Corporation, Santa Clara, CA, USA) and an NVIDIA RTX 4060 GPU(NVIDIA Corporation, Santa Clara, CA, USA). In the proposed LSTM-TD3 framework, the LSTM prediction network employs an encoder–decoder architecture, comprising two layers of LSTM networks with a hidden dimension of 64 for each layer. The input to the network is a historical trajectory sequence of vehicles with a length of 4, where the feature vector at each time step has six dimensions, including vehicle position, speed, direction, task size, and task interval information. The network incorporates a self-attention mechanism to capture long-range temporal dependencies, and its output is the vehicle state prediction for the next two time steps. For the TD3 network configuration, it consists of an actor network and two critic networks; each network contains two fully connected hidden layers with 600 and 400 neurons, respectively. The hidden layers utilize the ReLU activation function, while the output layer of the actor network adopts the Tanh function to constrain the action values within the range of $\left[-1,1\right]$. The specific configurations of key hyperparameters are presented in

Table 1. Hyperparameters of LSTM predictor and TD3 optimizer.

Parameters	Value
Discount factor, $\zeta$	0.999
Learning rate of LSTM predictor, ${\xi }^p$	0.001
Learning rate of actor network, ${\xi }^a$	0.005
Learning rate of critic network, ${\xi }^c$	0.005
Updated rate of target network, $\psi$	0.005
Mini-batch size, $|N_b|$	32
Delay update frequency of actor networks, d	2
Variance of explore noise, policy noise, $\sigma$	0.1

.

In order to verify that the algorithm can be used in a variety of scenarios, we set up two common scenarios, as shown in Figure 4. Map 1 is a 1000 m straight road with two RSUs evenly arranged on both sides of the road, and each RSU covers an area of 100 m. Map 2 is an area consisting of a 1000 m east–west lane and a 400 m north–south lane, with three RSUs deployed at (150, 100), (500, 300), and (850, 100). All vehicles are uniformly distributed along the road and travel in random directions with a predefined speed. The time slot length is 0.05 s, and at the commencement of each time slot, the vehicle randomly generates tasks according to the Poisson distribution, and the task arrival rate is $\lambda =\{0.2,0.4,0.6,0.8,1.0\}$. The network environment parameters of UAV-assisted VEC are shown in

Table 2. Environment parameters of UAV-assisted VEC.

Parameters	Description	Value
f	Carrier frequency	2100 MHz
C	Speed of light	$3\times {10}^8\mathrm{m}/\mathrm{s}$
$a,b$	A2G channel coefficient	12, 0.5
${\eta }_{Rayleigh}$	Rayleigh fading coefficient	0.707
${\eta }_{LoS},{\eta }_{NLoS}$	Additional path loss	1, 20
$W_R,W_U$	Total uplink bandwidth	20, 50 MHz
$p_n\left(t\right)$	Transmission power of vehicles	10 mW
$N_0$	AWGN power spectral density	−174 dBm/Hz
$D_n\left(t\right)$	Data size	[512, 1024] KB
$X_c$	Number of CPU cycles	1000 cycles/bit
$T_n^{max}$	Maximum tolerant latency of task	0.5 s
$F_n,F_R,F_U$	Computation capacity	1, 3, 8 GHz
$k_n$	Computation energy efficiency coefficient	${10}^{-27}$
$P_0,P_i$	Blade profile power and induced power in hovering status	79.8563, 88.6279 w
$U_{tip}$	Tip speed of rotor blade	120 m/s
$v_0$	Mean rotor-induced velocity in hovering status	4.03 m/s
$d_0,{\rho }_0$	Fuselage drag ratio, air density	$0.6{\mathrm{m}}^2,1.225{\mathrm{kg}/\mathrm{m}}^2$
$s_0,A_0$	Rotor solidity, rotor disc area	$0.05,0.503{\mathrm{m}}^2$
$v_u^{\max }$	UAV maximum flight speed	30 m/s
$E_U^{\max }$	Initial UAV energy	2000 J
$p_R,p_U$	Rental unit price	$2\times {10}^{-8}$, $4\times {10}^{-8}$/cycles

.

5.2. System Convergence

To verify the convergence of the proposed algorithm, we set the number of vehicles to 10, 30, and 50, respectively, and simulated the system’s convergence performance under sparse, normal, and congested conditions. For comparison, the TD3 algorithm was tested with 30 vehicles, and the results are presented in Figure 5.

We configured the training process with 1000 episodes, where each episode consists of 100 steps. As shown in Figure 5a, the average episode reward exhibits significant fluctuations around a low value in the initial stage of training. This is because the agent lacks sufficient environmental knowledge and relies on random actions for decision-making. With the progression of training, after accumulating an adequate amount of data samples, the reward increases rapidly. By approximately 200 episodes, the DRL network can achieve high and stable reward values. Additionally, as the number of vehicles N increases, the system cost rises accordingly, but the convergence speed remains largely unaffected.

It is worth noting that the performance fluctuation of LSTM-TD3 in the early stage of training (the first 200 rounds) is greater than that of TD3. This is because the prediction model has not fully converged, and the resulting prediction errors can lead to unstable decisions. With further training of the prediction model, this fluctuation will be significantly reduced, and stable high performance will eventually be achieved. The prediction accuracy of the prediction model is shown in Figure 5b.

As shown in Figure 6, we explored the impact of different learning rates on the performance of the LSTM-TD3 algorithm. In the experiment, parameters ${\xi }^a$, ${\xi }^c$, and ${\xi }^p$ were each set within the range of 0.0001 to 0.01, and various combinations of these parameters were tested. The results indicate that when the learning rates are too high, the training process becomes unstable. Conversely, when the learning rates are too low, not only does the convergence speed of training significantly slow down, but it also tends to get trapped in local optimal solutions. The experimental data shows that the algorithm achieves optimal performance when ${\xi }^a=0.005$, ${\xi }^c=0.005$, and ${\xi }^p=0.005$. Therefore, in subsequent experiments, we will adopt this set of learning rate parameters for related research.

5.3. Performance Evaluation

To comprehensively evaluate the performance advantage of our approach, we select the following methods as baselines:

• TD3: Standard TD3 algorithm without prediction mechanism, making decisions based only on the current observation state.
• DDPG: Deterministic Policy Gradient algorithm using a single critic network.
• LE (Local Execution): All tasks are performed locally.
• FO (Full Offloading): All tasks are completely offloaded to the RSU for execution.
• RO (Random Offloading): Computation tasks are randomly selected and executed locally, offloaded to RSUs or UAVs.

First, we conduct experiments on the two maps shown in Figure 4, where the number of vehicles N = 30. Figure 7 illustrates the experimental results. The system cost of the proposed LSTM-TD3 algorithm consistently remains lower than that of other strategies in both experimental scenarios, demonstrating the versatility and superiority of the LSTM-TD3 algorithm. In Map 1, the FO strategy incurs significantly higher costs than other approaches. The primary reason is that the insufficient number of RSUs fails to support the full offloading requirement of FO, leading to a rise in AoI and a surge in system costs. To facilitate the investigation of other indicators’ impact on the algorithm, subsequent experiments are conducted on MAP2.

To deeply analyze the impact of prediction accuracy on system performance, we designed a series of comparative experiments, and the results are shown in Figure 8. We compared three cases: (1) accurate prediction, using the real future state; (2) LSTM prediction, using our prediction model; and (3) no prediction, using only the current state.

The experimental results indicate that as the prediction horizon increases, system performance first improves and then declines. For the LSTM-TD3 framework, the optimal prediction horizon is two steps (equivalent to approximately 16 s), at which point the maximum performance improvement is achieved; beyond three steps, performance begins to decrease. This phenomenon arises because the accumulation of prediction errors over longer horizons leads to performance degradation. While the performance advantages of accurate predictions continue to grow with an extended prediction horizon—validating the theoretical value of forward-looking decision-making—in practical systems, prediction errors restrict the realization of this potential. These findings demonstrate that the quality of the prediction model is a critical factor influencing system performance, thereby confirming the necessity of incorporating prediction quality evaluation and adaptive fusion mechanisms.

We verified the performance of each algorithm under different strategies, as shown in Figure 9. First, we constructed four strategies: normal (${\gamma }_A$ = 0.33, ${\gamma }_E$ = 0.33, and ${\gamma }_P$ = 0.33), Time-sensitive (${\gamma }_A$ = 0.8, ${\gamma }_E$ = 0.1, and ${\gamma }_P$ = 0.1), energy-deficiency (${\gamma }_A$ = 0.1, ${\gamma }_E$ = 0.8, and ${\gamma }_P$ = 0.1), and fund-poor (${\gamma }_A$ = 0.1, ${\gamma }_E$ = 0.1, and ${\gamma }_P$ = 0.8). The experiment showed that in time-sensitive and energy-constrained scenarios, the performance of the LSTM-TD3 algorithm was significantly better than that of the TD3 algorithm. This is due to the forward-looking nature of the LSTM-TD3 algorithm, which can deploy drones in advance and plan the optimal path.

Figure 10 shows the performance comparison of each algorithm under different numbers of vehicles. The experimental range is from 5 vehicles to 50 vehicles to evaluate the adaptability of the system under different loads. The experimental results show that the total cost of each method increases with the number of vehicles, but the increase is different. Among them, the cost of LSTM-TD3 increases most slowly, reflecting its better scalability in high-load scenarios. When the number of vehicles is 30, the performance advantage of LSTM-TD3 over the original TD3 can reach up to 28.9%, indicating that prediction-driven decision-making is more significant in medium- and high-load scenarios. In low-load scenarios (10 vehicles), due to weak resource competition, the performance differences of each algorithm are small, and a simple strategy can achieve good results. In addition, as the number of vehicles increases, the performance of the LE strategy drops sharply, further verifying the necessity of computational offloading in high-load environments.

We study the impact of different task generation rates on system performance by adjusting the parameter $\lambda$ of the Poisson process, and the results are shown in Figure 11. The results show that the total cost of all methods increases significantly with the increase in task generation rate, but LSTM-TD3 increases the slowest. When $\lambda$ = 0.8, the average AoI of LSTM-TD3 is 24.9% lower than that of the original TD3. At medium and low task generation rates, the performance gap between LSTM-TD3 and TD3 is relatively small. However, as the task generation rate increases, the advantage of prediction-driven LSTM-TD3 becomes more obvious, which proves the importance of forward-looking decision-making in highly dynamic environments. The cost of the FO strategy rises sharply due to the increase in the number of tasks.

To evaluate the system’s adaptability to varying task sizes, we adjusted the task size range and observed system performance, as depicted in Figure 12. In the experiment, task sizes ranged from 0.5 MB to 4 MB. The results indicate that energy consumption and total costs for all methods increase significantly as task size grows. LSTM-TD3 demonstrates optimal performance across all task sizes, particularly in large-task scenarios. In such scenarios, LSTM-TD3 achieves the most substantial energy consumption advantage, reducing it by 21.7% compared to the original TD3. This is because the prediction mechanism allows the UAV to proactively relocate to vehicle-dense areas, thereby shortening transmission distances and lowering energy consumption.

Figure 13 illustrates an example demonstrating the forward-looking decision-making advantage of LSTM-TD3 in a specific scenario. Here, three vehicles are traveling at high speed from left to right and are on the verge of entering an RSU coverage blind spot. LSTM-TD3 anticipates this situation in advance, directing the UAV to relocate to the blind spot proactively for service preparation. In contrast, the original TD3 only moves the UAV after the vehicles have actually entered the blind spot, leading to evident decision latency.

This case intuitively demonstrates the core advantage of prediction-driven decision-making: it enables proactive adjustment of resource allocation based on future state predictions, thereby avoiding service quality degradation caused by reactive responses.

Integrating all experimental findings, our LSTM-TD3 framework exhibits significant performance advantages across diverse network conditions, particularly in highly dynamic environments and high-load scenarios. The prediction-driven forward-looking decision-making approach has pioneered a new research direction for resource optimization in UAV-assisted VEC networks.

6. Conclusions

In this paper, we propose a TD3 deep reinforcement learning framework integrated with LSTM-based trajectory prediction to address the resource optimization problem in the UAV-assisted Internet of Vehicles (IoV) edge computing environment. By predicting the future trajectories and task requirements of vehicles, this framework enables forward-looking resource allocation, effectively mitigating the decision lag issue in highly dynamic environments. The primary contributions of this study are outlined as follows: First, we developed a software-defined network (SDN)-driven UAV-assisted vehicular edge computing system model, covering communication, computing, energy consumption, and information timeliness models. This provides a comprehensive theoretical foundation for resource optimization. Second, for the first time, we organically integrated LSTM-based trajectory prediction with the TD3 reinforcement learning algorithm, designing a four-module framework comprising a vehicle trajectory predictor, state processor, TD3 optimizer, and decision fusion mechanism. Through a prediction–decision dual closed-loop mechanism, this framework realizes perception and optimization of future states, significantly enhancing the system’s adaptability to environmental changes. Third, we developed state smoothing and data augmentation techniques to enhance training stability and proposed a multi-objective optimization model integrating Age of Information (AoI), energy consumption, and resource rental costs, thereby comprehensively considering all aspects of system performance. Finally, through extensive experimental validations across diverse network scenarios, our LSTM-TD3 method outperforms existing approaches significantly in terms of total system cost, information freshness, and environmental adaptability. Notably, the advantages of prediction-driven decision-making are more pronounced in high-load and highly dynamic environments. When the number of vehicles $N=30$, the performance of LSTM-TD3 is improved by up to 24.9% compared with the original TD3; under high task generation rates, the average AoI is reduced by 31.5%; and in large-scale task scenarios, energy consumption is reduced by 21.7%.

In future work, we will focus on advancing research in several key directions: On the one hand, we plan to incorporate more sophisticated probabilistic modeling techniques and Bayesian reasoning frameworks, systematically exploring uncertainty quantification methods to overcome the limitations of existing prediction steps and accuracy. Simultaneously, by developing a system model more aligned with real-world scenarios (e.g., incorporating practical factors such as extreme weather and complex road networks), we will further enhance the system’s robustness in dynamic environments. On the other hand, multi-UAV collaboration mechanisms will be the core breakthrough, focusing on addressing issues of collaborative trajectory planning, load balancing, and interference management in large-scale complex scenarios to build a collaborative system adaptable to highly dynamic environments. Additionally, security research is equally essential; we plan to leverage adversarial network techniques to strengthen the system’s capability to resist malicious interference and abnormal data, ensuring the stable operation of critical applications. Finally, we will also prioritize advancing the lightweight design of models and the development of efficient architectures, reducing computational overhead via techniques such as neural network pruning and knowledge distillation, improving real-time response capabilities in resource-constrained scenarios, and facilitating the framework’s transition to practical deployment.