Joint Scheduling and Placement for Vehicular Intelligent Applications Under QoS Constraints: A PPO-Based Precedence-Preserving Approach

Abstract

The increasing demand for low-latency, computationally intensive vehicular applications, such as autonomous navigation and real-time perception, has led to the adoption of cloud–edge–vehicle infrastructures. These applications are often modeled as Directed Acyclic Graphs (DAGs) with interdependent subtasks, where precedence constraints enforce causal ordering while allowing concurrency. We propose a task offloading framework that decomposes applications into precedence-constrained subtasks and formulates the joint scheduling and offloading problem as a Markov Decision Process (MDP) to capture the latency–energy trade-off. The system state incorporates vehicle positions, wireless link quality, server load, and task-buffer status. To address the high dimensionality and sequential nature of scheduling, we introduce DepSchedPPO, a dependency-aware sequence-to-sequence policy that processes subtasks in topological order and generates placement decisions using action masking to ensure partial-order feasibility. This policy is trained using Proximal Policy Optimization (PPO) with clipped surrogates, ensuring stable and sample-efficient learning under dynamic task dependencies. Extensive simulations show that our approach consistently reduces task latency, energy consumption and QOS compared to conventional heuristic and DRL-based methods. The proposed solution demonstrates strong applicability to real-time vehicular scenarios such as autonomous navigation, cooperative sensing, and edge-based perception.

1. Introduction

With the rapid advancement of mobile communication technologies, cloud computing, and distributed artificial intelligence, modern network systems are experiencing a paradigm shift towards decentralized, latency-sensitive computing [1]. Within this shift, Mobile Edge Computing (MEC) has become a key paradigm, relocating compute, storage, and networking capabilities from distant data centers to access points near users [2]. Executing analytics and decisions at the edge shortens round-trip times and yields faster system response. MEC now supports applications across smart manufacturing, industrial IoT, telemedicine, and intelligent transportation systems (ITS) [3,4,5].

Among these applications, Intelligent Transportation Systems (ITS) have gained particular attention due to the increasing demand for real-time services and the proliferation of connected and autonomous vehicles [6,7]. Vehicular Edge Computing (VEC), a specialized form of MEC tailored for vehicular networks, plays a central role in supporting the computational requirements of in-vehicle services such as autonomous navigation, augmented reality dashboards, cooperative perception, and traffic-aware route planning [8]. These vehicular applications typically involve data-intensive and latency-sensitive workloads, which must be processed under strict Quality of Service (QoS) constraints [9,10,11].

While traditional task offloading techniques in VEC assume that computational workloads are independent and can be flexibly partitioned or distributed, real-world vehicular applications often consist of a series of interdependent subtasks with intricate data and control dependencies [12]. Such applications are best modeled using Directed Acyclic Graphs (DAGs) [13], where each node represents an atomic subtask and directed edges represent precedence constraints and data flows, for example, DAG-based task structures are commonly seen in autonomous driving pipelines (e.g., perception → localization → planning → control), video analytics (e.g., decoding → object detection → event reasoning), and traffic management systems (e.g., data collection → fusion → prediction → decision) [5].

In a typical VEC architecture, vehicles interact with nearby Roadside Units (RSUs) equipped with MEC servers and may also access remote cloud servers through Base Stations (BSs) [14]. However, the dynamic topology of vehicular networks—characterized by high mobility, intermittent connectivity, and fluctuating bandwidth—introduces substantial uncertainty into the task offloading process [15,16]. Additionally, the coverage range of RSUs is geographically limited, and MEC servers deployed at the edge possess only limited computational and communication resources [17]. As a result, efficient task offloading in such environments must not only respect the structural dependencies among subtasks but also adapt to link variations, resource constraints, and deadline requirements.

Despite extensive research efforts in this field, most existing works fall short in two aspects. (1) They oversimplify task models by assuming subtask independence, which limits their applicability to real-world vehicular services [18,19,20]; (2) they lack robust mechanisms to adapt to environmental dynamics, such as vehicle movement and RSU availability, resulting in degraded system performance and reliability [17,21,22].

To address these limitations, this paper investigates a dependency-aware task offloading framework, a hierarchical VEC system composed of three layers: the cloud layer, the edge layer, and the vehicle layer. In this framework, vehicular tasks are modeled as DAGs, where nodes represent atomic subtasks and edges denote data dependencies. Each subtask executes either onboard the vehicle, at a nearby MEC node, or in the remote cloud. The framework jointly optimizes the execution order and offloading decisions of subtasks, considering vehicle mobility, link availability, computation capacity, and task deadlines.

We formulate the joint scheduling and offloading problem as a Markov Decision Process (MDP), which captures the evolution of system states including vehicle positions, network connectivity, task queue status, and resource availability. To tackle the high-dimensional, complex action space and non-convex optimization landscape, we propose DepSchedPPO (Dependency-Aware Task Scheduling using Proximal Policy Optimization), an actor–critic paradigm deep reinforcement learning algorithm. DepSchedPPO integrates a sequence-to-sequence (Seq2Seq) graph encoder to extract DAG inside structural features, enabling the policy network to make informed decisions on subtask scheduling and placement. The critic network estimates long-term rewards and guides policy updates, ensuring convergence and robustness in a dynamic environment.

The main contributions of this paper are summarized as follows:

• Hierarchical VEC Architecture with DAG Task Modeling: We propose a cloud–edge–vehicle architecture that uses DAGs to represent subtasks, incorporating both sequential and parallel execution for vehicular applications.
• Seq2Seq-Based Policy Network for Dependency-Aware Scheduling: Unlike conventional DRL methods that use feedforward DNNs, we design a novel Seq2Seq policy network to capture task dependencies and historical scheduling decisions, improving offloading accuracy and system performance.
• PPO-Based Optimization for QoS Maximization: We adopt Proximal Policy Optimization (PPO) to ensure stable learning and maximize the quality of service (QoS) in dynamic vehicular environments.
• Comprehensive Performance Evaluation: Extensive simulations demonstrate that DepSchedPPO outperforms existing methods in reducing task latency and energy consumption under diverse network conditions.

Paper outline: Section 2 summarizes related work. Section 3 introduces the model and formulates the optimization problem. Section 4 details the DepSchedPPO. Section 5 provides simulation results and performance evaluations. Finally, Section 6 closes with directions for follow-up studies.

2. Related Work

In vehicular edge computing (VEC) environments, efficient task offloading has become a prominent research topic in recent years. Existing studies have proposed a variety of offloading decision-making strategies based on different optimization objectives, including minimizing delay [23,24,25], energy consumption [26,27], system cost [28], and maximizing quality of service (QoS) [29,30] in single-objective optimization scenarios, as well as multi-objective joint optimization problems [31,32]. These studies provide a wealth of solutions in the task offloading domain; however, for applications with data or execution dependencies between tasks and for achieving resource-awareness and policy learning in dynamic vehicular networks, systematic approaches are still lacking. Therefore, this paper reviews the related work from the perspectives of VEC architectures and task offloading.

2.1. Vehicular Edge Computing Architectures

VEC systems generally adopt a multi-layer distributed architecture, consisting of the cloud layer, vehicle layer and edge layer. The traditional VEC architecture primarily relies on roadside units (RSUs) equipped with edge servers to respond to computation demands, combined with cloud servers for support [33,34,35]. For example, Xun et al. [33] constructed a three-tier vehicle–edge–cloud collaborative architecture that uses cloud-trained models to evaluate driving behavior at the edge. Liu et al. [36] investigated cooperative task computing and on-demand resource allocation in VEC, proposing a multi-resource cooperative framework involving central cloud, RSUs, and mobile vehicles. Dai et al. [34] studied the heterogeneous computing collaboration problem between MEC servers and the cloud, optimizing task allocation to minimize overall task delay. They considered service migration in multi-server scenarios and used asynchronous deep reinforcement learning to optimize the task offloading problem, minimizing the weighted sum of user utility.

Another model is the vehicle-assisted edge computing (VAEC) architecture [18,19,20], which leverages idle computational resources from nearby vehicles to perform task execution, thereby reducing communication overhead and improving system responsiveness [37,38]. For instance, Han et al. [39] proposed a VAEC framework to distribute delay-sensitive tasks to neighboring vehicles and edge servers for joint processing. Feng et al. [40] introduced an innovative partial reverse offloading (PRO) strategy to utilize vehicle resources, reducing the load on VEC and minimizing system delay through optimized reverse offloading decisions and resource allocation. Ma et al. [41] presented a three-tier VEC framework that utilizes the vast underused resources of parked and moving vehicles in urban areas to build a collaborative distributed computing architecture, effectively addressing resource constraints and expanding VEC service coverage. Qin et al. [42] addressed the issue of limited computational resources in vehicles by leveraging idle resources from other vehicles and applying Lyapunov optimization to decompose the task offloading problem into vehicle-side offloading and cooperative vehicle-side resource allocation, achieving optimization of short-term decisions and long-term queue delays.

Although the above architectures have made progress in service migration, task scheduling and heterogeneous resource management, there are still challenges in practical deployment, such as communication instability caused by high mobility, channel interference, and limited service coverage. Furthermore, some VAEC systems face risks of privacy leakage and insufficient device security authentication. Considering system robustness and privacy protection requirements, current research tends to favor using fixed infrastructure (RSUs and MEC servers) for task offloading scheduling.

2.2. Task Offloading

With the rapid development of VEC, vehicular computational tasks increasingly become complex, often comprising multiple interdependent subtasks. These subtasks exhibit strong coupling in data flows, execution order, and timing constraints, as seen in typical applications such as image recognition, path planning, and decision making in autonomous driving, frame processing and behavior detection in video analytics, and multi-sensor data fusion [43]. Such dependencies are often modeled as directed acyclic graphs (DAGs), where nodes represent atomic subtasks and edges represent data or control dependencies.

Compared with independent task offloading, dependency-aware task offloading is more complex. On the one hand, the execution order of subtasks must strictly follow the DAG’s topology to preserve logical consistency; on the other hand, offloading involves multiple computational nodes (vehicles, edge servers, and cloud servers) and must consider node computation capacity, network bandwidth, transmission delay, and vehicle mobility to formulate optimal offloading and scheduling strategies. Since task scheduling for DAG structures has been proven to be NP-hard [44], traditional heuristic algorithms (e.g., HEFT [45], ITAGS [46]) or metaheuristic algorithms (e.g., GSA [47], QPSO [48]) may perform well for small-scale tasks but suffer from rapidly increasing scheduling time as the number of tasks grows, limiting their application in large-scale and computation-intensive scenarios. Moreover, most existing dependency-aware task offloading algorithms focus on single DAG scheduling or static scheduling, whereas the complexity of dynamic vehicular applications in VEC makes handling multi-vehicle, multi-task dependency scenarios a key research challenge.

As deep reinforcement learning (DRL) has been widely applied to dynamic optimization problems, more research has introduced DRL into dependency-aware task offloading to overcome the limitations of traditional methods in responsiveness and generalization under dynamic, complex environments. DRL learns policies through environment interaction, enabling end-to-end modeling and adaptive optimization of task scheduling and resource allocation. Lin et al. [49] proposed the SA-DQN algorithm, modeling the task offloading problem as a Markov decision process (MDP) and introducing task queue depth to measure subtask priority, combined with simulated annealing to optimize task selection strategies. Liu et al. [50] targeted a vehicle–edge–cloud collaborative architecture and adopted an improved DDPG algorithm with a task execution delay-based reward function to achieve minimum-delay strategies in heterogeneous computing environments. Liu et al. [51] aimed to minimize the deadline violation ratio and proposed the DVRMO-MM scheme, which incorporates task migration and merging by decomposing the dependency-aware task offloading problem into two subproblems. Zhao et al. [52] presented a Stackelberg game-based dependency-aware task offloading and resource pricing framework (SDOP) that models the interaction between SDN controllers and vehicles, considering task dependencies and using backward induction with gradient ascent plus a genetic algorithm (GAPG) to solve for Nash and Stackelberg equilibria, significantly improving both controller and vehicle utility while achieving better delay and energy performance in various scenarios. Luo et al. [53] proposed a joint task offloading and resource allocation strategy in an MEC environment with vehicle–roadside infrastructure sensor (RIS) collaboration, incorporating delay, energy consumption, and tolerance for interruptions of incremental information (T3I) into a unified decision-making framework via Lyapunov optimization, effectively improving offloading accuracy and system stability in asymmetric resource scenarios. Considering the structural characteristics of DAG tasks, some studies have further integrated graph neural networks (GNNs) into DRL architectures. Sun et al. [54] proposed the GRLO framework, which uses GNNs to encode task graph structures, extracting task dependencies via message passing, and combines this with an Actor–Critic strategy to output scheduling actions and evaluate performance. Yan et al. [55] employed an Actor–Critic structure, analyzing graph paths to construct a Critic evaluation function and introducing a graph neural structure to optimize the Actor’s offloading policy, significantly improving task completion rate and system stability. Zhang et al. [56] proposed the DSO algorithm for dependency-aware task offloading and service placement optimization, combining improved multi-agent Q-learning (IMAQL) with a greedy scheduling algorithm to maximize application hit ratio and improve QoS in multi-application scenarios. Additionally, for continuous action space scenarios (e.g., fine-grained task offloading, dynamic resource allocation), a number of studies based on DDPG, PPO, and other policy gradient methods [57,58] have emerged. These methods can handle complex state–action spaces while adapting to dynamic VEC environments, gradually becoming mainstream in this research direction.

2.3. Summary

In summary, Vehicular Edge Computing (VEC) systems have made significant strides in architecture design, resource collaboration, and task scheduling. A variety of methods have been proposed to optimize key objectives such as latency, energy consumption, and system cost. Additionally, there is a growing trend to integrate Deep Reinforcement Learning (DRL) into task offloading processes, aiming to enhance decision-making efficiency and adaptability in dynamic, unpredictable environments.

However, when dealing with large-scale, complex dependency-aware tasks, existing DRL-based methods still suffer from several limitations. First, these methods often struggle with insufficient state modeling, which prevents the accurate representation of dynamic vehicular systems and their interactions. Furthermore, the task dependency features, which are critical for efficient offloading and scheduling, are often underutilized. This leads to suboptimal convergence efficiency and robustness, particularly in continuous action spaces. Another significant gap is the lack of a unified and scalable framework for collaborative optimization of heterogeneous resources, which are essential for achieving optimal performance in real-world, multi-vehicle VEC scenarios. Additionally, task topology features, which can further optimize offloading decisions, remain inadequately exploited.

The studies reviewed in this section highlight these challenges. Table 1 provides a comparative analysis of the key literature in VEC task offloading, showcasing how each approach addresses these challenges and identifying their respective limitations. Notably, existing work either overlooks or fails to fully address the integration of task dependencies, real-time QoS optimization, and dynamic resource allocation, which are crucial for achieving optimal system performance in heterogeneous environments.

Table 1. Comparison of key literature in vehicular edge computing (VEC) and task offloading.

Study	Objective	Methodology	Key Findings	Research Gap/Contribution of This Work
Xun et al. [33]	VEC architecture	Cloud–edge–vehicle collaboration	Vehicle behavior evaluation using cloud-trained models	Does not address task dependencies or dynamic offloading based on mobility.
Liu et al. [36]	Cooperative task computing	Multi-resource cooperative framework	Effective task allocation, but limited to static resources	No dynamic adaptation based on vehicle movement and network conditions.
Dai et al. [34]	Task allocation with MEC	Asynchronous deep reinforcement learning	Minimizes task delay via cloud–edge collaboration	Focuses only on delay minimization, while also optimizing energy and QoS.
Han et al. [39]	Reduce latency using edge computing	Distributed task offloading to neighboring vehicles	Reduced latency by leveraging nearby vehicle resources	Does not address energy consumption or QoS optimization.
Zhao et al. [52]	Task offloading in dynamic networks	Stackelberg game-based model	Improved utility for vehicles and controllers	Does not incorporate task dependencies or adjust based on urgency.
Ma et al. [41]	Multi-vehicle task offloading	Lyapunov optimization for task offloading	Delays reduced via cooperative task offloading between vehicles	Does not handle task dependencies or energy efficiency.
Feng et al. [40]	Reverse offloading strategy	Reverse offloading decision optimization	Optimizes reverse offloading to reduce VEC load	Neglects task dependency modeling and its impact on QoS and latency.
Qin et al. [42]	Resource allocation in VEC	Cooperative vehicle-side resource allocation	Efficient offloading and resource sharing	Does not model task dependencies or optimize QoS.
Sun et al. [54]	GNN for task offloading	GNNs and reinforcement learning	Encodes task dependencies for offloading decisions	Lacks QoS optimization and dynamic adaptation to vehicle conditions.
Yan et al. [55]	Task offloading using actor–critic and GNNs	Actor–Critic with GNN task modeling	Enhances task offloading efficiency using GNNs	Focuses on static task offloading, whereas our approach adapts dynamically.
Zhang et al. [56]	Multi-agent Q-learning for offloading	Q-learning with greedy scheduling	Improved QoS by optimizing task offloading decisions	Does not model task dependencies or handle dynamic adjustments.
Our Work	Dependency-aware task offloading optimization	Proximal Policy Optimization (PPO) with dependency modeling	Reduces task latency and energy, optimizing QoS in dynamic environments	Integrate dependency-aware task offloading with reinforcement learning for VEC, optimizing latency, energy, and QoS.

Our work contributes to filling these gaps by proposing a dependency-aware task offloading optimization framework using Proximal Policy Optimization (PPO). This framework not only incorporates a dynamic, task dependency model but also leverages task topology features, improving convergence efficiency and robustness while providing a scalable solution for collaborative resource optimization across heterogeneous vehicles.

3. System Model

The vehicular edge computing (VEC) architecture employed in this study is illustrated in Figure 1, which models a road scenario. The system is organized into three hierarchical layers: cloud, edge and vehicle layers. The vehicle layer consists of vehicles equipped with limited onboard computational capabilities. These vehicles are able to communicate with base stations (BSs) and roadside units (RSUs) via 5G/LTE networks and dedicated wireless interfaces. The edge layer includes multiple roadside nodes deployed at various locations along the road. Each node integrates an RSU with limited wireless coverage and an associated Mobile Edge Computing (MEC) server that provides moderate computing and storage resources. Notably, the coverage areas of RSUs are non-overlapping to prevent signal interference. The cloud layer forms the top tier, pooling large-scale compute resources accessed from BSs via cabled backhaul. These cloud servers offer centralized computing support for vehicle applications when edge resources are insufficient or unavailable. When a task is generated onboard the vehicle, it can either be executed locally or offloaded to the MEC server or the cloud server, depending on resource availability, task urgency, and network conditions.

Figure 1. VEC system architecture.

The VEC architecture considered in this study, as shown in Figure 1, models a road scenario with three layers:

• Vehicle layer—on-road vehicles with limited computing capacity, communicating with BS and RSU.
• Edge layer—roadside nodes each combining an RSU (non-overlapping coverage to avoid interference) and an MEC server with moderate computing and storage capabilities.
• Cloud layer—large-scale compute resources accessed from BSs via cabled backhaul, providing centralized resources when edge capacity is insufficient.

When a task is generated, it may run locally or be offloaded to MEC or cloud servers. according to resource availability, task urgency, and network conditions.

Referring to [59], the vehicular edge computing (VEC) system in this paper operates in discrete time slots with uniform duration. At each time slot, a MEC server is capable of communicating with vehicles located within its current coverage region. The base station (BS) provides full-area wireless coverage and enables all vehicles to access cloud computing services when necessary. For vehicles outside the RSU service range, computation tasks must be offloaded to the cloud.

Let $\mathcal{K}=\{1,2,\dots ,k\}$ denote the RSU set, and $\mathcal{N}=\{1,2,\dots ,n\}$ denote the vehicle set in the system. As illustrated in Figure 2, the navigation application is modeled as a task graph, and nodes represent subtasks and directed edges describe dependency relationships. The label inside each node indicates the subtask’s local execution time.

Figure 2. Topology of vehicle navigation task.

The execution process proceeds as follows. A user inputs a destination into the navigation interface, prompting the controller module to retrieve the vehicle’s current GPS coordinates. Subsequently, three parallel subtasks are initiated to acquire location data, route options, and traffic conditions along the planned route. These subtasks are designed for concurrent execution, and their dependencies are represented by a Directed Acyclic Graph (DAG). A subtask must be executed when all of its predecessor subtasks have completed, as their outputs act as its inputs.

Based on this dependency representation, task offloading must respect both sequential constraints and parallelism among subtasks. Each vehicular task generated within a time slot is formulated as a DAG, denoted by $G=(L,E)$, where L is the set of subtasks and E is the set of directed edges. For vehicle i’s m-th task, the e-th subtask is represented as $Q_{m,e}^i$. Each subtask is described by the tuple $Q_{m,e}^i=<D_{m,e}^i,C_{m,e}^i,T_{i,m,e}^{max}>$, where $D_{m,e}^i$ denotes the input data size, $C_{m,e}^i$ indicates the computational workload (in CPU cycles), and $T_{i,m,e}^{max}$ represents the maximum allowable execution latency. Table 2 summarizes the definitions of the key symbols used in this paper.

Table 2. Key notations used in this paper.

Symbol	Description
$K=\{1,2,\dots ,k\}$	Set of RSUs (Road Side Units)
$N=\{1,2,\dots ,n\}$	Set of vehicles
$Q_{m,e}^i=⟨D_{m,e}^i,C_{m,e}^i,T_{i,m,e}^{\max }⟩$	The e-th subtask of the m-th task from vehicle i
$D_{m,e}^i$	Data volume of subtask $Q_{m,e}^i$
$C_{m,e}^i$	Computational complexity of subtask $Q_{m,e}^i$
$T_{i,m,e}^{\max }$	Maximum tolerable latency of task $Q_m^i$
$R_{mec,up}$/$R_{mec,down}$	Uplink and downlink trans rate between vehicle and MEC(V2M/M2V)
$R_{cloud,up}$/$R_{cloud,down}$	Uplink and downlink trans rate between vehicle and cloud(V2C/C2V)
$P_{local}$/$P_{exc}^{mec}$	Computing power cost of the vehicle terminal and MEC server
$P_{up}^{mec}$/$P_{down}^{mec}$	Transmission power cost of the MEC uplink and downlink channels
$P_{up}^{cloud}$/$P_{down}^{cloud}$	Transmission power cost of the cloud uplink and downlink channels
$f_{local}$/$f_{mec}$/$f_{cloud}$	Computing capability of the vehicle, MEC, and cloud

3.1. Problem Assumptions

In VEC environment, task offloading decisions are influenced by dynamic factors, like vehicle speed, network bandwidth, geographic location, and signal strength. However, considering all these factors simultaneously is highly complex in real-world scenarios. Building upon the foundations established previously, the following assumptions are made in this paper to facilitate tractable analysis of the proposed problem:

• The considered tasks are dependency-aware, representing intelligent vehicular applications composed of multiple interdependent subtasks. Each subtask is atomic, meaning it cannot be further divided and must be executed in a single stage without interruption.
• The vehicular edge computing system operates in a time-division manner, providing services to all vehicles in discrete time slots. It is assumed that tasks generated by vehicles can be completed within a single time slot.
• Due to data dependencies among subtasks, the output of a predecessor subtask may serve as input for its successor. Consequently, the return of intermediate results must be considered during task offloading to ensure correct execution of downstream subtasks.
• The dependency structure among subtasks may permit partial parallelism, meaning some subtasks can be executed concurrently. This increases the number of subtasks potentially offloaded to remote servers. However, due to the limited bandwidth capacity of MEC server communication channels, offloaded subtasks may experience queuing delays when the channel is occupied.

3.2. Problem Modeling

(1) Task Model

Unlike independent tasks, dependent tasks consist of multiple subtasks that can be executed locally or offloaded to edge or cloud servers. These subtasks are interdependent, where the output of one may serve as the input to another. We map each task to a DAG $G=\left(L,E\right)$; L enumerates the subtasks and E contains directed edges capturing execution dependencies. For vehicle i, the m-th task is denoted by $Q_m^i$, and its e-th subtask is denoted by $Q_{m,e}^i$. A directed edge $\left(Q_{m,e}^i,Q_{m,k}^i\right)$ indicates that subtask $Q_{m,k}^i$ can only start after the completion of subtask $Q_{m,e}^i$.

Figure 3 illustrates an example where the main task $Q_m^i$ is subdivided into nine subtasks: $(Q_{m,1}^i,Q_{m,2}^i,Q_{m,3}^i,Q_{m,4}^i,Q_{m,5}^i,Q_{m,6}^i,Q_{m,7}^i,Q_{m,8}^i,Q_{m,9}^i)$. In this task partitioning, $Q_{m,1}^i$ acts as the start node, and $Q_{m,9}^i$ serves as the end node, which only begins execution after all its predecessor subtasks are completed. $Q_{m,3}^i$ is the predecessor for both $Q_{m,4}^i$ and $Q_{m,5}^i$. Conversely, $Q_{m,4}^i$ and $Q_{m,5}^i$ are successors of $Q_{m,3}^i$, meaning $Q_{m,4}^i$ and $Q_{m,5}^i$ can only start after $Q_{m,3}^i$ finishes. Additionally, subtasks like $Q_{m,2}^i$, $Q_{m,3}^i$, or $Q_{m,7}^i$, $Q_{m,8}^i$, can execute in parallel. Each subtask $Q_{m,e}^i$ is represented by a triple $Q_{m,e}^i=⟨D_{m,e}^i,C_{m,e}^i,T_{i,m,e}^{max}⟩$, where $D_{m,e}^i$ denotes the e-th subtask’s input data size, $C_{m,e}^i$ represents the amount of computational resources required to complete subtask $Q_{m,e}^i$, and $T_{i,m,e}^{max}$ denotes the maximum allowable delay for subtask $Q_{m,e}^i$.

Figure 3. Example of DAG.

Based on the DAG, this paper introduces the Ready Time (RT) and Finish Time ($FT$) to describe subtask dependencies. For subtask $Q_{m,e}^i$, the Ready Time $R{T}_{m,e}^i$ is the earliest moment it can start, determined by the completion of all its immediate predecessors. $F{T}_{m,e}^i$ denotes the moment the subtask completes execution. According to these definitions, the Ready Time of subtask $Q_{m,e}^i$ is:

$${\mathrm{RT}}_{m,e}^{\mathrm{i}}=\left\{\begin{array}{cc}{\max }_{Q_{m,e^{\prime }}^i\in \mathrm{pre}\left(Q_{m,e}^i\right)}F{T}_{m,e^{\prime }}^i& \mathrm{pre}\left(Q_{m,e}^i\right)\ne \varnothing ,\\ S{T}_m^i& \mathrm{pre}\left(Q_{m,e}^i\right)=\varnothing .\end{array}\right.$$

(1)

Equation (1) indicates that when a subtask has no direct predecessors (i.e., $\mathrm{pre}\left(Q_{m,e}^i\right)=\varnothing$), its Ready Time is the start time of the entire in-vehicle application, $S{T}_m^i$. Otherwise, when subtask $Q_{m,e}^i$ is not the start task, its Ready Time depends on its direct predecessor nodes $\mathrm{pre}\left(Q_{m,e}^i\right)$, equaling the maximum Finish Time among all its direct predecessor tasks.

(2) Computation Model

Local Computation Model:

Within time slot t, $Q_{m,e}^i$ is the subtask’s local execution on vehicle i, and then we can write

$$T_{i,m,e}^{local}={\displaystyle \frac{C_{m,e}^i}{f_{local}}},$$

(2)

where $C_{m,e}^i$ is the computational complexity, and $f_{local}$ represents the local processing capability of the vehicle.

Due to limitations in the vehicle’s onboard computational resources, this study assumes that only one subtask can be executed locally within a time slot. The subtask’s Earliest Start Time ($EST$) denotes the earliest time it can begin local execution as

$$ES{T}_{m,e}^{i,local}=R{T}_{m,e}^i+A{T}_{m,e}^{i,local},$$

(3)

where $A{T}_{m,e}^{i,local}$ is the Available Time of subtask $Q_{m,e}^i$ for local computation on the vehicle terminal, signifying the earliest time the vehicle has idle computational resources for execution:

$$A{T}_{m,e}^{i,local}=\left\{\begin{array}{c}0,L\left(i\right)=\varnothing \\ {\max }_{Q_{m,e^{\prime }}^i\in L\left(\mathrm{i}\right)}F{T}_{m,e^{\prime }}^{\mathrm{i},l\mathrm{local}}-{\mathrm{RT}}_{m,e}^{\mathrm{i}},L\left(\mathrm{i}\right)\ne \varnothing \end{array}\right.$$

(4)

where $L\left(i\right)$ is the queue of tasks waiting to be executed locally. As shown, the Available Time of subtask $Q_{m,e}^i$ is the maximum Finish Time among all remaining subtasks in the queue.

The Finish Time of subtask $Q_{m,e}^i$ executed locally on the vehicle terminal is

$$F{T}_{m,e}^{i,\mathrm{local}}=ES{T}_{m,e}^{i,\mathrm{local}}+T_{i,m,e}^{\mathrm{local}}.$$

(5)

The local energy consumption for computing subtask $Q_{m,e}^i$ on the vehicle terminal is

$$\begin{array}{c}{E}_{i,m,e}^{\mathrm{local}}=P_{\mathrm{local}}·T_{i,m,e}^{\mathrm{local}},\end{array}$$

(6)

where $P_{\mathrm{local}}$ denotes the power consumption.

MEC Computation Model:

The time required to offload a computation task to MEC for execution mainly consists of three phases:

• Transmission Phase: The period to send the computation task to MEC. Due to limited channel bandwidth, tasks might experience waiting time before accessing an idle channel. Thus, the delay here consists of both the waiting time for channel availability and the wireless transmission duration.
• Execution Phase: The duration taken by MEC to process the assigned task.
• Download Phase: The time needed to deliver the computation results back to the vehicle terminal. As the returned data size is much smaller than the uploaded input, channel bandwidth is typically negligible here.

For subtask $Q_{m,e}^i$, the time spent in the transmission, execution, and download phases can be expressed by Equations (7), (8), and (9), respectively, as

$$\begin{array}{cc}\hfill T_{m,e,j}^{i,\mathrm{trans}\_\mathrm{up}}& ={\displaystyle \frac{D_{m,e}^i}{R_{\mathrm{mec},\mathrm{up}}}},\hfill \end{array}$$

(7)

$$\begin{array}{cc}\hfill T_{m,e,j}^{i,\mathrm{exc}}& ={\displaystyle \frac{C_{m,e}^i}{f_{\mathrm{mec}}}},\hfill \end{array}$$

(8)

$$\begin{array}{cc}\hfill T_{m,e,j}^{i,\mathrm{trans}\_\mathrm{down}}& ={\displaystyle \frac{D_{m,e}^{i,t}}{R_{\mathrm{mec},\mathrm{down}}}},\hfill \end{array}$$

(9)

where $T_{m,e,j}^{i,\mathrm{trans}\_\mathrm{up}}$ represents the upload delay for sending subtask $Q_{m,e}^i$ to MEC server j, $T_{m,e,j}^{i,\mathrm{exc}}$ represents the execution time of subtask $Q_{m,e}^i$ on MEC server j, and $T_{m,e,j}^{i,\mathrm{trans}\_\mathrm{down}}$ represents the time required from MEC server to vehicle terminal to return the computation results. $R_{\mathrm{mec},\mathrm{up}}$ and $R_{\mathrm{mec},\mathrm{down}}$ denote the upload and download speeds of the V2M link, respectively.

When subtask $Q_{m,e}^i$ is offloaded to an MEC server, its Earliest Eligible Transmission time ($EET$) is

$$EE{T}_{m,e,j}^{i,\mathrm{edge}}=R{T}_{m,e}^i+A{T}_{m,e,j}^{i,\mathrm{edge}},$$

(10)

where $R{T}_{m,e}^i$ is the Ready Time of subtask $Q_{m,e}^i$, and $A{T}_{m,e,j}^{i,\mathrm{edge}}$ represents the Available Time of the V2M upload channel. It is defined as

$$A{T}_{m,e}^{i,\mathrm{edge}}=\left\{\begin{array}{c}0,B\left(j\right)=\varnothing \\ {\max }_{Q_{m,e^{\prime }}^i\in B\left(j\right)}EE{T}_{m,e,j}^{\mathrm{i},\mathrm{edge}}-{\mathrm{RT}}_{m,e}^{\mathrm{i}},B\left(j\right)\ne \varnothing ,\end{array}\right.$$

(11)

where $B\left(j\right)$ represents the set of subtasks queued for transmission from vehicle i to MEC server j at time $R{T}_{m,e}^i$. The Earliest Reception Time ($ERT$) at the edge server j is then

$$ER{T}_{m,e,j}^{i,\mathrm{edge}}=EE{T}_{m,e,j}^{i,\mathrm{edge}}+T_{m,e,j}^{i,\mathrm{trans}\_\mathrm{up}}.$$

(12)

When subtask $Q_{m,e}^i$ is offloaded to an MEC server for execution, its Finish Time on the MEC server is

$$F{T}_{m,e}^{i,\mathrm{edge}}=ER{T}_{m,e,j}^{i,\mathrm{edge}}+T_{m,e,j}^{i,\mathrm{exc}}+T_{m,e,j}^{i,\mathrm{trans}\_\mathrm{down}}.$$

(13)

The energy consumption for offloading subtask $Q_{m,e}^i$ to the MEC server can be expressed as

$$E_{m,e,j}^{i,\mathrm{mec}}=P_{\mathrm{up}}^{\mathrm{mec}}·T_{m,e,j}^{i,\mathrm{trans}\_\mathrm{up}}+P_{\mathrm{exc}}^{\mathrm{mec}}·T_{m,e,j}^{i,\mathrm{exc}}+P_{\mathrm{down}}^{\mathrm{mec}}·T_{m,e,j}^{i,\mathrm{trans}\_\mathrm{down}},$$

(14)

where $P_{\mathrm{up}}^{\mathrm{mec}}$ and $P_{\mathrm{down}}^{\mathrm{mec}}$ are the power consumption rates for the MEC server’s upload and download channels, respectively, and $P_{\mathrm{exc}}^{\mathrm{mec}}$ is the power consumption rate for computation on the MEC server.

Cloud Computation Model:

The time required to offload a computation task to the cloud server for execution also consists of three phases:

• Transmission Phase: The period required to transmit the computation task to the cloud server. Due to abundant communication resources at the cloud server, tasks do not need to wait for an idle channel; transmission starts immediately upon readiness.
• Execution Phase: The duration taken by the cloud server to execute the task. Sufficient computational resources are available on the cloud server; subtasks can be executed directly upon arrival.
• Download Phase: The time taken for the computation results to the vehicle terminal device. As with the MEC model, the returned data size is assumed to be small, so channel bandwidth limitations are ignored.

Let $f_{\mathrm{cloud}}$ represent the computational capability of the cloud server. The transmission time for subtask $Q_{m,e}^i$ to the cloud server (V2C) is denoted $T_{m,e,\mathrm{cloud}}^{i,\mathrm{trans}\_\mathrm{up}}$, the time for results to return is denoted $T_{m,e,\mathrm{cloud}}^{i,\mathrm{trans}\_\mathrm{down}}$, and the computation time on the cloud server is denoted $T_{m,e,\mathrm{cloud}}^{i,\mathrm{exc}}$. Specifically, the transmission and execution delays are calculated as follows:

$$\begin{array}{cc}\hfill T_{m,e,\mathrm{cloud}}^{i,\mathrm{trans}\_\mathrm{up}}& ={\displaystyle \frac{D_{m,e}^i}{R_{\mathrm{cloud},\mathrm{up}}}},\hfill \end{array}$$

(15)

$$\begin{array}{cc}\hfill T_{m,e,\mathrm{cloud}}^{i,\mathrm{exc}}& ={\displaystyle \frac{C_{m,e}^i}{f_{\mathrm{cloud}}}},\hfill \end{array}$$

(16)

$$\begin{array}{cc}\hfill T_{m,e,\mathrm{cloud}}^{i,\mathrm{trans}\_\mathrm{down}}& ={\displaystyle \frac{D_{m,e}^{i,t}}{R_{\mathrm{cloud},\mathrm{down}}}}.\hfill \end{array}$$

(17)

When subtask $Q_{m,e}^i$ is offloaded to the cloud, the Earliest Reception Time ($ERT$) at the cloud server $ER{T}_{m,e}^{i,\mathrm{cloud}}$ and the Finish Time ($FT$) on the cloud server $F{T}_{m,e}^{i,\mathrm{cloud}}$ can be expressed as

$$\begin{array}{cc}\hfill ER{T}_{m,e}^{i,\mathrm{cloud}}& =R{T}_{m,e}^i+T_{m,e,\mathrm{cloud}}^{i,\mathrm{trans}\_\mathrm{up}},\hfill \end{array}$$

(18)

$$\begin{array}{cc}\hfill F{T}_{m,e}^{i,\mathrm{cloud}}& =ER{T}_{m,e}^{i,\mathrm{cloud}}+T_{m,e,\mathrm{cloud}}^{i,\mathrm{exc}}+T_{m,e,\mathrm{cloud}}^{i,\mathrm{trans}\_\mathrm{down}}.\hfill \end{array}$$

(19)

Due to cloud server’s abundant computational resources, the energy consumption associated with computation execution on the cloud is not considered by the system. Therefore, the energy consumption when offloading subtask $Q_{m,e}^i$ to the cloud server is

$$E_{m,e}^{i,\mathrm{cloud}}=P_{\mathrm{up}}^{\mathrm{cloud}}·T_{m,e,\mathrm{cloud}}^{i,\mathrm{trans}\_\mathrm{up}}+P_{\mathrm{down}}^{\mathrm{cloud}}·T_{m,e,\mathrm{cloud}}^{i,\mathrm{trans}\_\mathrm{down}}.$$

(20)

Considering the Finish Time ($FT$) and energy consumption (E) of subtask $Q_{m,e}^i$ when executed locally, on MEC, or in the cloud, we define $a_{m,e}^i=\{0,1,2\}$ as the offloading decision variable, where $a_{m,e}^i=0$ denotes local processing, $a_{m,e}^i=1$ represents execution on MEC, and $a_{m,e}^i=2$ corresponds to cloud execution. For a given task $Q_m^i$, the overall completion time is determined by the largest $FT$ among its subtasks, while the total energy consumption equals the sum of E for all subtasks.

$$\begin{array}{cc}\hfill T_m^i& =\underset{Q_{m,e}^i\in Q_m^i}{\max }\left\{F{T}_{m,e}^{i,\mathrm{local}},F{T}_{m,e}^{i,\mathrm{edge}},F{T}_{m,e}^{i,\mathrm{cloud}}\right\},\hfill \end{array}$$

(21)

$$\begin{array}{cc}\hfill E_m^i& =\sum _{Q_{m,e}^i\in Q_m^i}\left({\mathbb{I}}_{(a_{m,e}^i=0)}{E}_{i,m,e}^{\mathrm{local}}+{\mathbb{I}}_{(a_{m,e}^i=1)}{E}_{m,e,j}^{i,\mathrm{mec}}+{\mathbb{I}}_{(a_{m,e}^i=2)}{E}_{m,e}^{i,\mathrm{cloud}}\right),\hfill \end{array}$$

(22)

where ${\mathbb{I}}_{\left(condition\right)}$ denotes the indicator function, which returns 1 when the specified condition holds true and 0 otherwise.

3.3. Problem Formulation

We quantify system-level QoS as a weighted combination of latency and energy consumption. The aggregate latency and energy over all tasks are given by:

$$\begin{array}{cc}\hfill T_{\mathrm{total}}& =\underset{i\in N,m\in M}{\max }{T}_m^i\hfill \end{array}$$

(23)

$$\begin{array}{cc}\hfill E_{\mathrm{total}}& =\sum _{i\in N}\sum _{m\in M}{E}_m^i\hfill \end{array}$$

(24)

A higher QoS represents a better quality of service provided by the system to the vehicle users. Therefore, lower overall system delay and energy consumption correspond to higher user QoS. The QoS is defined as

$${\mathrm{QoS}}_{a_{m,e}^i}=\alpha ·{\displaystyle \frac{{\sum }_{i\in N,m\in M}{T}_{i,m,e}^{\mathrm{local}}-T_{\mathrm{total}}}{{\sum }_{i\in N,m\in M}{T}_{i,m,e}^{\mathrm{local}}}}+\beta ·{\displaystyle \frac{{\sum }_{i\in N,m\in M}{E}_{i,m,e}^{\mathrm{local}}-E_{\mathrm{total}}}{{\sum }_{i\in N,m\in M}{E}_{i,m,e}^{\mathrm{local}}}}.$$

(25)

Based on the above models, the optimization goal of this paper is to maximize the system’s QoS. The optimization problem can be formulated as

$$\begin{array}{cc}\hfill & G:\max {\mathrm{QoS}}_{a_{m,e}^i}\hfill \\ & \mathrm{s}.\mathrm{t}.\hfill \end{array}$$

(26)

$$\begin{array}{cc}& \mathrm{C}1:\alpha +\beta =1\hfill \end{array}$$

(26a)

$$\begin{array}{cc}& \mathrm{C}2:\alpha \in [0,1],\beta \in [0,1]\hfill \end{array}$$

(26b)

$$\begin{array}{cc}& \mathrm{C}3:a_{m,e}^i\in \left\{0,1,2\right\}\forall i,m,e\hfill \end{array}$$

(26c)

where C1 and C2 are constraints on the energy consumption weighting coefficients, reflecting the balance between latency and energy consumption within the optimization goal, with C2 specifying the valid range for the weights. C3 is the offloading decision constraint; the value of $a_{m,e}^i$ indicates the offloading decision for subtask $Q_{m,e}^i$. When $a_{m,e}^i=0$, the subtask $Q_{m,e}^i$ is executed locally on the vehicle terminal; when $a_{m,e}^i=1$, the subtask is offloaded to an MEC server; when $a_{m,e}^i=2$, the subtask is offloaded to the cloud server.

4. Algorithm Design

The optimization objective of this paper is to maximize the QOS during system operation. From a system perspective, this is achieved by minimizing task delay and energy consumption as much as possible. This paper introduces a dependency-aware vehicular task offloading method based on PPO to solve the target optimization problem (26). Building upon the DRL framework for VEC proposed in the previous paper, this section first describes the MDP model for the problem, then introduces the neural network architecture used in this reinforcement learning framework, and finally details the DepSchedPPO algorithm based on the PPO framework along with its training process.

4.1. Markov Decision Process Elements

According to the definition of the dependency task model, for the offloading decision of vehicle subtask $Q_{m,e}^i$, it depends on the network environment, vehicle state, edge server state, task-specific information, and previous task scheduling results.

State Space: When scheduling subtask $Q_{m,e}^i$, the system’s state space is defined as

$$s_t=\left\{p\left(t\right),q\left(t\right),G\left(t\right),A_{1:e}\left(t\right)\right\},$$

(27)

where $p\left(t\right)$ represents the coordinate positions of mobile edge computing servers, $q\left(t\right)$ is the coordinate position matrix of vehicles, $G\left(t\right)$ is the task model represented as a Directed Acyclic Graph (DAG) after modeling, containing task dependencies along with task-specific data size and computational complexity information. $A_{\{1:e\}}\left(t\right)$ is the partial offloading decision matrix, including the scheduling decision set for subtasks from $Q_{m,1}^i$ to $Q_{m,e}^i$.

While a DAG can model dependencies between subtasks, it is difficult to directly input into a neural network. Therefore, the DAG corresponding to task dependencies needs to be embedded and converted into vector representation. To ensure that the embedded DAG sequence still satisfies the task characteristics of dependency tasks, it is necessary to determine the order of subtasks for different DAG sequences. Based on this, a priority queue model for subtasks is first established to determine the operational order for different applications. We rank subtasks using their latest completion time and latest execution time. Specifically, the Latest Completion Time (LCT) for $Q_{m,e}^i$ is

$$LC{T}_{m,e}^i=\left\{\begin{array}{cc}{T}_{i,m}^{\max },& Q_{m,e}^i\in \mathrm{end}\left(Q_{m,e}^i\right),\\ \min \left(\underset{Q_{m,e^{\prime }}^i\in \mathrm{succ}\left(Q_{m,e}^i\right)}{min}\left(LC{T}_{m,e^{\prime }}^i-\underset{\mathrm{loc}\in N\cup K\cup C}{min}{T}_{m,e^{\prime }}^{i,\mathrm{loc}}\right),T_{i,m,e}^{\max }\right),& Q_{m,e}^i\notin \mathrm{end}\left(Q_m^i\right),\end{array}\right.$$

(28)

where $T_{m,e^{\prime }}^{i,\mathrm{loc}}$ represents the delay of subtask $Q_{m,e^{\prime }}$ when offloaded to local, edge server, or cloud server $\mathrm{loc}\in \{N\cup K\cup C\}$ without considering queuing time. The $LCT$ is obtained through iterative calculation starting from the end node. Based on the $LCT$, the Latest Execution Time ($LET$) of the subtask is

$$LE{T}_{m,e}^i=LC{T}_{m,e}^i-\underset{\mathrm{loc}\in N\cup K\cup C}{min}{T}_{m,e,\mathrm{loc}}^{i,\mathrm{exc}},$$

(29)

where $T_{m,e,\mathrm{loc}}^{i,\mathrm{exc}}$ represents the execution time of the subtask at offloading destination loc, and $LE{T}_{m,e}^i$ indicates the urgency of the subtask. A higher $LE{T}_{m,e}^i$ value indicates lower urgency. Therefore, for all subtasks, they are sorted in ascending order of $LE{T}_{m,e}^i$ to determine the operational order, i.e., the subtask priority queue.

After converting the task’s DAG into a prioritized task sequence, embedding operations are performed to facilitate neural network training. The embedding vector contains three elements: subtask-related information including estimated runtime delay, an index vector of predecessor subtasks, and an index vector of successor subtasks. The initial vector is padded with −1 and then fed into the neural network to train for offloading decisions.

Action Space: Subtask $Q_{m,e}^i$ may be executed locally, on MEC, or in the cloud. The agent’s action determines the subtask’s offloading destination, with the action space defined as

$$a_t=\left\{O_1\left(t\right),\dots ,O_n\left(t\right)\right\},$$

(30)

where $O_i\left(t\right)$ represents the task decision matrix of vehicle i at step t, composed of elements $a_{m,e}^i=0/1/2$. The numbers 0, 1, and 2 indicate whether the offloading location is local, edge server, or cloud server, respectively.

Reward: The optimization objective defined in this paper is to maximize system quality of service. To optimize QoS, we take the cumulative return in (26) as the agent’s reward

$$r_t=\alpha \left(t\right)·{\displaystyle \frac{{\sum }_{i\in N,m\in M}{T}_{i,m,e}^{\mathrm{local}}-T_{\mathrm{total}}}{{\sum }_{i\in N,m\in M}{T}_{i,m,e}^{\mathrm{local}}}}+\beta \left(t\right)·{\displaystyle \frac{{\sum }_{i\in N,m\in M}{E}_{i,m,e}^{\mathrm{local}}-E_{\mathrm{total}}}{{\sum }_{i\in N,m\in M}{E}_{i,m,e}^{\mathrm{local}}}}.$$

(31)

4.2. Seq2Seq Neural Network Architecture

After modeling the abstracted MDP elements of the optimization problem, this section elaborates on the construction of the Seq2Seq neural network for the DepSchedPPO algorithm. Considering the characteristics of dependency tasks, a DAG is used to model the dependencies between subtasks. As mentioned in the previous subsection, to facilitate neural network training, the prioritized DAG task sequence is embedded. The embedded DAG sequence is $\{{\mathbf{x}}_{\mathbf{1}},...,{\mathbf{x}}_{\mathbf{g}},{\mathbf{x}}_{\mathbf{e}},...,{\mathbf{x}}_{\mathbf{E}}\}$, where E is the number of subtasks. According to the MDP definition, for a DAG task graph, the policy $\pi \left(a_e\right|s_e)$ represents the probability of selecting action $a_e$ for subtask $Q_{m,e}^i$ in state $s_e=\{p,q,G,A_{1:e-1}\}$. Let $\pi \left(A_{1:e}\right|\{p,q,G\})$ be the probability of choosing an offloading plan $A_{1:e}$ for a task graph G, given the environment state $\{p,q,G\}$. By applying the chain rule of probability, we can express this as a product of sequential decisions as

$$\pi \left(A_{1:e}\right|\{p,q,G\})=\prod _{e=1}^E\pi .$$

(32)

To approximate the policy of the MDP defined in Equation (32) and evaluate the value function $V^{\pi }\left(s_e\right)$ of the current state, unlike traditional DNN neural networks used in conventional reinforcement learning structures, this paper employs a Seq2Seq network architecture as the neural network for the DepSchedPPO algorithm. This is because traditional DNNs are feedforward networks without feedback loops, where data passes sequentially through input, hidden, and output layers, with each layer containing multiple neurons transformed using nonlinear activation functions. This makes DNNs more suitable for data without dependency relationships in input information. However, according to Equation (32), for task graph G, its offloading plan $A_{1:e}$ depends on the previous offloading plan $A_{1:e-1}$.

The Seq2Seq neural network architecture defined in this paper is shown in Figure 4. This architecture consists of two recurrent neural networks: an Encoder network and a Decoder network. The embedded DAG sequence $\{{\mathbf{x}}_{\mathbf{1}},\dots ,{\mathbf{x}}_{\mathbf{g}},\dots ,{\mathbf{x}}_{\mathbf{e}},\dots ,{\mathbf{x}}_{\mathbf{E}}\}$ is input into the Encoder network. Through forward propagation, all input sequences are encoded into a unified feature vector, denoted as $\mathbf{c}$. Subsequently, combined with an attention mechanism, the Decoder network decodes the feature vector $\mathbf{c}$ output by the Encoder to produce the output sequence. The detailed processing of the Seq2Seq neural network is as follows.

Figure 4. Seq2Seq neural network architecture.

(1) Encoder Network Processing: For the defined DAG sequence $\{{\mathbf{x}}_{\mathbf{1}},\dots ,{\mathbf{x}}_{\mathbf{g}},\dots ,{\mathbf{x}}_{\mathbf{e}},\dots ,{\mathbf{x}}_{\mathbf{E}}\}$, the Encoder processes the sequence. At each time step, the Encoder receives a vector ${\mathbf{x}}_{\mathbf{g}}$ and outputs a hidden state ${\mathbf{H}}_{\mathbf{g}}$.

$${\mathbf{H}}_{\mathbf{g}}\left(\mathrm{Encoder}\right)={\phi }^e\left({\mathbf{H}}_{\mathbf{g}-\mathbf{1}}\left(\mathrm{Encoder}\right),{\mathbf{x}}_{\mathbf{g}}\right)$$

(33)

Using the attention mechanism, define $at{t}_{g,e}$ as the relevance between the hidden state ${\mathbf{H}}_{\mathbf{g}}$ at stage g of the Encoder and the hidden state at stage e of the Decoder ${\mathbf{H}}_{\mathbf{e}}^{\prime }$. Therefore, the feature vector ${\mathbf{c}}_{\mathbf{e}}$ is represented as the weighted sum of all ${\mathbf{H}}_{\mathbf{g}}$ over $at{t}_{g,e}$:

$${\mathbf{c}}_{\mathbf{e}}=\sum _{g=1}^{E}at{t}_{g,e}·{\mathbf{H}}_{\mathbf{g}}\left(\mathrm{Encoder}\right)$$

(34)

The attention distribution $at{t}_{g,e}$ is calculated as follows, where ${\mathbf{H}}_{\mathbf{e}}^{\prime }$ is the hidden layer state of the Decoder at stage e, and $s(H,H^{\prime })$ is the attention scoring function, computed using the dot product model:

$$at{t}_{g,e}={\displaystyle \frac{\exp \left(s\left({\mathbf{H}}_{\mathbf{g}},{\mathbf{H}}_{\mathbf{e}-\mathbf{1}}^{\prime }\right)\right)}{{\sum }_{j=1}^E\exp \left(s\left({\mathbf{H}}_{\mathbf{j}},{\mathbf{H}}_{\mathbf{e}-\mathbf{1}}^{\prime }\right)\right)}}$$

(35)

(2) Decoder Network Processing: After the Encoder outputs the feature vector $\mathbf{c}$, the Decoder decodes it. At each stage, the Decoder produces a scheduling decision output $a_e$ until the complete output queue is generated. The hidden layer node state of the Decoder is computed as follows:

$${\mathbf{H}}_{\mathbf{e}}^{\prime }\left(\mathrm{Decoder}\right)={\phi }^d\left({\mathbf{H}}_{\mathbf{e}-\mathbf{1}}^{\prime }\left(\mathrm{Decoder}\right),{\mathbf{c}}_{\mathbf{e}-\mathbf{1}},a_{e-1}\right)$$

(36)

Specifically, the Decoder operates as follows. At each time step e, based on the previous scheduling decision output $a_{e-1}$, the current hidden state ${\mathbf{H}}_{\mathbf{e}}^{\prime }$, and the context vector ${\mathbf{c}}_{\mathbf{e}}$ provided by the Encoder, it computes a logits vector. Then, a fully connected layer applies the softmax function to convert these logits into a probability distribution.

$$\pi \left(a_e\right|s_e)={\displaystyle \frac{\exp \left({\mathrm{logits}}_{a_e}\right)}{{\sum }_e\exp \left({\mathrm{logits}}_e\right)}}$$

(37)

Here, ${\mathrm{logits}}_{a_e}$ is the logit value corresponding to output $a_e$, and the denominator is the sum of the exponentials of all possible output logits, ensuring that the sum of all output probabilities is 1. Additionally, to approximate the value network of the MDP process, the value of the state $V^{\pi }\left(s_e\right)$ is output through another fully connected layer. Finally, the Decoder selects the output $a_e$ with the highest probability as the output for this stage.

4.3. DepSchedPPO Algorithm Architecture

This section proposes the Dependency-Aware Task Scheduling Algorithm based on Proximal Policy Optimization (DepSchedPPO) tailored for dependency task offloading characteristics. The architecture of DepSchedPPO is shown in Figure 5.

Figure 5. DepSchedPPO algorithm architecture.

Actor–Critic reinforcement learning algorithms contain two core components: an Actor network responsible for behavioral decisions and a Critic network for value assessment. The learning process starts with the Actor network, which selects an action a in a given state s based on a determined probability p. After executing action a, the Actor receives a reward r and a new state $s^{\prime }$. This transition information is then passed to the Critic network. The Critic calculates a label value called Temporal Difference Error (${\mathrm{TD}}_{\mathrm{error}}$), representing the deviation between the Critic’s prediction and the actual outcome. This ${\mathrm{TD}}_{\mathrm{error}}$ is used to adjust the Critic network and is also fed back to the Actor network to assist in updating its parameters.

As shown in Figure 5, the proposed DepSchedPPO framework employs an Actor–Critic structure. The Actor component consists of two separate neural networks, $Acto{r}_1$ and $Acto{r}_2$, which generate the current policy ${\pi }_{\theta }\left(a_t\right|s_t)$ and the previous policy ${\pi }_{{\theta }_{\mathrm{old}}}\left(a_t\right|s_t)$, respectively. To regulate policy updates, the Actor computes the probability ratio $r_t\left(\theta \right)$ between the current and previous policies, ensuring that policy changes remain within a controlled range to preserve training stability. The computation of $r_t\left(\theta \right)$ is expressed as:

$$r_t\left(\theta \right)={\displaystyle \frac{{\pi }_{\theta }\left(a_t\right|s_t)}{{\pi }_{{\theta }_{\mathrm{old}}}\left(a_t\right|s_t)}}$$

(38)

After computing the ratio $r_t\left(\theta \right)$, the objective function $L^{\mathrm{CLIP}}\left(\theta \right)$ clips the value of $r_t\left(\theta \right)·{\widehat{A}}_t$ to avoid instability caused by excessive updates, enabling smooth policy optimization. The network’s objective function is computed as:

$$L^{\mathrm{CLIP}}\left(\theta \right)={\mathbb{E}}_t\left[\min \left(r_t\left(\theta \right)·{\hat{A}}_t,\mathrm{clip}(r_t\left(\theta \right),1-ϵ,1+ϵ){\widehat{A}}_t\right)\right]$$

(39)

Here, ${\mathbb{E}}_t$ denotes the expectation, computed by averaging over samples at time step t. The term $r_t\left(\theta \right)$ denotes the probability ratio between the updated policy and the previous policy, while ${\hat{A}}_t$ refers to the estimated advantage, acting as a learning signal that measures the benefit to the mean performance. The clipping operation $\mathrm{clip}(r_t\left(\theta \right),1-ϵ,1+ϵ)$ constrains $r_t\left(\theta \right)$ within $[1-ϵ,1+ϵ]$, thereby preventing excessive changes to the policy; here, $ϵ$ specifies the clipping threshold.

To lower the variance in gradient estimation, the algorithm adopts the Generalized Advantage Estimator (GAE) [60] as the advantage function, defined as:

$${\hat{A}}_t={\eta }_t+\left(\gamma \lambda \right){\eta }_{t+1}+\cdots +{\left(\gamma \lambda \right)}^{T-t+1}{\eta }_{T-1},$$

(40)

where $\gamma$ is the discount factor reducing the current value of future rewards, and $\lambda$ is the GAE parameter acting as a smoothing factor weighting bias and variance. ${\eta }_t$ is the Temporal Difference (TD) error at time step t, representing the difference between the predicted state value function and the observed return. Minimizing TD error improves the accuracy of the value function prediction, enhancing learning performance. The TD error is calculated as:

$${\eta }_t=r_t+\gamma V\left(s_{t+1}\right)-V\left(s_t\right)$$

(41)

Due to parameter sharing, the loss function combines the policy gradient error term and value function. Therefore, the loss function of DepSchedPPO combines the clipped policy gradient part $L_t^{\mathrm{CLIP}}\left(\theta \right)$, the entropy bonus $S\left[{\pi }_{\theta }\right]\left(s_t\right)$ [61], and the value function loss part $L^{\mathrm{V}}\left(\theta \right)$. As defined in Equation (39), $L_t^{\mathrm{CLIP}}\left(\theta \right)$ prevents instability from large policy changes. The value function loss $L^{\mathrm{V}}\left(\theta \right)$ improves prediction accuracy of future rewards. The entropy bonus $S\left[{\pi }_{\theta }\right]\left(s_t\right)$ encourages the policy to explore more possible actions, promoting better long-term decisions and avoiding local optima. Combining these balances exploration and exploitation during learning, optimizing long-term policy performance. The loss function is expressed as:

$$L^{\mathrm{CLIP}+\mathrm{V}+\mathrm{S}}\left(\theta \right)={\widehat{\mathbb{E}}}_t\left[L_t^{\mathrm{CLIP}}\left(\theta \right)-c_1·L^{\mathrm{V}}\left(\theta \right)+c_2·S\left[{\pi }_{\theta }\right]\left(s_t\right)\right]$$

(42)

Here, $c_1$ and $c_2$ are hyperparameters that balance the contribution of the value loss and the entropy term in the overall loss function.

The value loss component $L^{\mathrm{V}}\left(\theta \right)$ is evaluated via the Mean Squared Error (MSE) [62], with the objective of reducing the discrepancy between predicted outputs and target values

$$L^{\mathrm{V}}\left(\theta \right)={(V_{\theta }\left(s_t\right)-V_{\theta }\left(s_{t+1}\right))}^2,$$

(43)

where $V_{\theta }\left(s_t\right)$ represents the value network’s prediction for state $s_t$, and $V_{\theta }\left(s_{t+1}\right)$ is the target value prediction for the next state.

The entropy bonus is added to the total loss function to encourage exploration. Entropy measures uncertainty; in reinforcement learning, high entropy indicates a more uniform probability distribution over possible actions in a given state, encouraging exploration of more action combinations and avoiding premature convergence to local optima. The entropy bonus term $S\left[{\pi }_{\theta }\right]\left(s_t\right)$ is defined as:

$$S\left[{\pi }_{\theta }\right]\left(s_t\right)=-\sum _a{\pi }_{\theta }\left(a\right|s_t)log{\pi }_{\theta }\left(a\right|s_t)$$

(44)

4.4. DepSchedPPO Training and Testing Algorithms

Based on the DepSchedPPO architecture above, this section further presents the DepSchedPPO training and testing algorithms.

Algorithm 1 outlines the DepSchedPPO training process. It trains the Seq2Seq neural network to output scheduling decisions for the DAG task sequence corresponding to each dependency task. The process first constructs the Seq2Seq neural network structure for the agent’s actor and critic networks and initializes a trajectory set D to record system state–action transition pairs and trajectories. Lines 3–20 describe the training loop. During each episode iteration, priority embedding operations are performed on all vehicle DAG tasks according to Equations (28) and (29). The embedded DAG sequence is then fed into the agent. Lines 6–10 depict the sampling process of the Actor network, where it samples actions for the DAG sequence, executes them in the system environment, and stores the resulting trajectory data in set $D_i$. Lines 11–20 update the Actor and Critic networks. Specifically, a random subset of trajectory segments is selected from $D_i$. Using the Critic network, the advantage estimate ${\widehat{A}}_t$ is computed via Equations (40) and (41), and the target state value estimate $V_{\theta }\left(s_{\mathrm{target}}\right)$ is output and stored in $D_i$. The Actor network then calculates the ratio $r_t\left(\theta \right)$ between new and old policies using Equation (38), computes the Actor’s optimization objective $L^{\mathrm{CLIP}}$ via Equation (39), and updates the Actor. Finally, the combined objective $L^{\mathrm{CLIP}+\mathrm{V}+\mathrm{S}}\left(\theta \right)$ is optimized using Equations (42), (43) and (44), and the Critic network is updated. The synchronized policy network parameters ${\theta }_{\mathrm{old}}$ are returned.

Algorithm 2 describes the DepSchedPPO testing algorithm, simulating the operation of the vehicular edge system and testing the effectiveness of DepSchedPPO. The training algorithm processes vehicle driving states and outputs policy network parameters $\theta$.

The testing algorithm then uses these trained parameters $\theta$ to make scheduling decisions for each vehicle’s DAG tasks during system operation in each time slot, computing delay, energy consumption, and system quality of service.

Algorithm 1: DepSchedPPO training algorithm

Algorithm 2: DepSchedPPO validating algorithm

5. Simulation and Result Analysis

5.1. Experimental Setup

The simulation process is crucial to demonstrate the effectiveness of our proposed dependency-aware task offloading framework. We employ a vehicular edge computing (VEC) environment to simulate the task offloading process under varying network conditions and vehicular mobility. The simulation involves multiple steps to ensure comprehensive evaluation of the algorithm’s performance.

The experimental setup utilizes Python 3.9 with TensorFlow on an Ubuntu 22.04.1 system, equipped with an Intel Xeon Gold 6148 CPU (2.40 GHz) and an NVIDIA GeForce RTX 3090 GPU. The VEC network follows a three-tier cloud–edge–vehicle framework, comprising centralized cloud servers, distributed mobile edge servers, and multiple vehicular nodes. The simulation is conducted on a straight segment of a bidirectional highway. In this setup, the cloud server’s base station covers the entire 2D rectangular map, while mobile edge servers are positioned randomly along the roadside, each providing circular wireless coverage. Vehicles travel at constant speeds within the base station’s coverage area.

In vehicular edge computing systems, intelligent vehicle applications can be modeled as Directed Acyclic Graphs (DAGs) with various topological structures. For example, navigation applications consist of tasks with multiple linear dependencies, while autonomous driving systems involve more complex interdependencies. To train a general and effective task offloading strategy, diverse task profiles and dependencies of different intelligent vehicle applications must be considered. Since existing application datasets contain limited application information [63], this experiment uses the DAG Generator [64] to generate DAGs for heterogeneous vehicular applications. The DAG Generator constructs DAG properties through parameters such as $fat$, $density$, and $ccr$, where: $fat$ controls the width and height of the DAG, $density$ measures the connectivity level of nodes in the DAG, $ccr$ represents the ratio of communication cost to computation cost.

Following references [65,66], the DAG generation parameters are configured as follows: $fat$ values: $\{0.3,0.4,0.5,0.6,0.7,0.8\}$; $density$ values: $\{0.3,0.4,0.5,0.6,0.7,0.8\}$; $ccr$ values: $\{0.2,0.3,0.4\}$ (since most intelligent vehicle applications are computation-intensive). For task information in DAGs: number of subtasks per DAG: $[10,50]$, subtask data size: $[10,50]$ KB, computational complexity: $[100,500]$ Gcycles. For reinforcement learning hyperparameters, both Actor and Critic networks use the Seq2Seq architecture, which includes encoder neural network and decoder neural network, and both networks are 2-layer LSTMs with 256 hidden layer nodes.

To ensure optimal performance, the proposed algorithm relies on the fine-tuning of several key hyperparameters that directly influence its efficiency and stability. Specifically, the clipping range ($ϵ=0.2$) is chosen to prevent large policy updates, which could otherwise lead to instability during training. The learning rate ($lr$ = $5\times {10}^{-4}$) is carefully set to control the speed of convergence. Additionally, the Generalized Advantage Estimator (GAE) factor ($ϵ=0.95$) and the discount factor ($\gamma$ = 0.99) are selected to strike an effective balance between bias and variance in reward estimation, which is crucial for maintaining the algorithm’s robustness across dynamic task dependencies and varying network conditions in vehicular edge computing (VEC) environments. Furthermore, the value loss coefficient ($c_1=0.5$) and the entropy coefficient ($c_2=0.01$) play a critical role in balancing exploration and exploitation, enhancing the long-term decision-making capabilities of the algorithm. The architecture of the neural network is also optimized, with the number of layers and hidden units adjusted to align with the computational capabilities of the VEC system.

Table 3 and Table 4 detail the experimental environment parameters and deep learning configurations.

Table 3. Experimental parameter settings.

Parameter	Value	Parameter	Value
Training vehicles $N_1$	500	Test vehicles $N_2$	100
V2M_up $R_{\mathrm{mec},\mathrm{up}}$	9 Mbps	V2M_down $R_{\mathrm{mec},\mathrm{down}}$	9 Mbps
V2C_up $R_{\mathrm{cloud},\mathrm{up}}$	6 Mbps	V2C_down $R_{\mathrm{cloud},\mathrm{down}}$	6 Mbps
Subtask data $D_{m,e}^i$	[10, 50] KB	Subtask complexity C	[100, 500] Gcycles
Subtask output $D_{m,e}^{i,t}$	[10, 50] KB	Vehicle terminal $f_{\mathrm{local}}$	1 GHz
Edge server $f_{\mathrm{mec}}$	10 GHz	Cloud server $f_{\mathrm{cloud}}$	80 GHz

Table 4. Deep learning hyperparameter settings.

Parameter	Value	Parameter	Value
Neural network layers	2	Hidden layer units	256
Clip range $ϵ$	0.2	Learning rate $lr$	$5\times {10}^{-4}$
GAE factor $\lambda$	0.95	Discount factor $\gamma$	0.99
Value loss coefficient $c_1$	0.5	Entropy coefficient $c_2$	0.01

The simulation reflects dynamic vehicular mobility, fluctuating network bandwidth, and varying link quality. Vehicles travel at constant speeds within a defined area, with edge servers along the road and the base station ensuring full-area coverage for cloud connectivity, allowing vehicles outside the edge server’s range to offload tasks to the cloud.

• Task Offloading Simulation: Task graphs for each vehicle simulate offloading under three execution environments: local vehicle processing, MEC servers, and cloud servers. The DepSchedPPO algorithm makes offloading decisions based on real-time data, including network state, vehicular position, and available resources.
• Dynamic Mobility and Network Conditions: The simulation incorporates changing vehicle mobility and network conditions, which affect task offloading decisions due to bandwidth fluctuations and edge server status.
• Evaluation Metrics: We evaluate the algorithm using the following. Task Latency: Time from task initiation to result return. Energy Consumption: Total energy used by the vehicle, MEC, and cloud. Quality of Service (QoS): A composite metric balancing latency and energy consumption.

Results are compared with baseline methods, including heuristic and reinforcement learning-based offloading strategies, highlighting the effectiveness of our approach in reducing latency and energy consumption while maintaining high QoS under dynamic conditions.

This detailed simulation process demonstrates the practicality and robustness of our method, addressing challenges in dynamic task dependencies and real-time resource management in VEC systems.

5.2. Result and Analysis

Based on the above experimental setup, this subsection first verifies the effectiveness of the DepSchedPPO algorithm, then compares its algorithmic efficiency with baseline algorithms through controlled variable experiments. The impact of the number of DAG subtasks, V2M transmission rates, energy–delay trade-off weights, and vehicle computational capabilities on system delay, energy consumption, and quality of service (QoS) is evaluated.

5.2.1. Effectiveness of DepSchedPPO

As mentioned in Section 4.3, DepSchedPPO adopts an Actor–Critic framework. The Actor network adjusts policy update magnitudes using Equations (38) and (39) to compute the ratio $r_t\left(\theta \right)$ between new and old policies and the clipped policy objective $L^{\mathrm{CLIP}}\left(\theta \right)$, thereby avoiding training instability from excessive updates and enabling smooth policy optimization. Additionally, according to Equation (42), the overall loss function $L^{\mathrm{CLIP}+\mathrm{V}+\mathrm{S}}\left(\theta \right)$ consists of three components: $L^{\mathrm{CLIP}}\left(\theta \right)$: Stabilizes policy updates; $S\left[{\pi }_{\theta }\right]\left(s_t\right)$: Policy entropy bonus encourages exploration of under-evaluated actions, preventing premature convergence to local optima; $L^{\mathrm{V}}\left(\theta \right)$: Value function loss minimizes prediction errors in the Critic network, providing accurate feedback for Actor decisions.

The combined loss function optimizes long-term returns while ensuring stability and enhancing overall policy efficacy through exploration. Beyond the conventional average reward metric, this section uses policy loss, entropy bonus, and value loss to validate DepSchedPPO’s effectiveness.

Figure 6a shows the Value Loss during the training process. The decreasing trend indicates that the Critic network is improving its ability to estimate the value function over time. Initially, the sharp drop reflects rapid learning from random initialization, while the stabilization after 600 epochs suggests that the model has converged to a stable and reliable value estimate, which is essential for guiding the offloading decisions effectively. Figure 6b illustrates the Policy Loss over time. The high initial volatility reflects the exploration–exploitation trade-off as the policy network tries to learn optimal offloading decisions. As the training progresses, the policy loss decreases, indicating that the algorithm is stabilizing its decision-making process. After 600 epochs, the policy loss stabilizes, suggesting that the algorithm has achieved a balance between exploration of new strategies and exploiting learned behaviors. Figure 6c shows entropy bonus, which serves to encourage exploration. A higher entropy value indicates a more uniform action distribution, meaning that the policy is exploring a wider range of possible decisions. As the training progresses, the entropy stabilizes, which shows that the policy network has converged to a more refined decision-making strategy, balancing exploration with exploitation. Figure 6d confirms performance improvement through rising average rewards, with convergence after 600 epochs indicating mastery of effective offloading decisions.

Figure 6. Value loss, policy loss, policy entropy bonus, and average reward during DepSchedPPO training.

5.2.2. Efficiency of DepSchedPPO

To evaluate DepSchedPPO’s efficiency, we compare it against seven baselines under varying experimental parameters. Metrics include average delay, energy consumption, and QoS:

• ASTOTL [67]: All subtasks execute locally on vehicles (baseline for potential offloading benefits).
• ASTOTM [67]: All subtasks offloaded to MEC servers.
• ASTOTC [60]: All subtasks offloaded to cloud servers (high compute but long transmission).
• RANDOS [60]: Random offloading strategy (lower-bound reference).
• RRSA [68]: Round-Robin scheduling algorithm is suitable for fairness scenarios, and the subtasks are alternately offloaded to the vehicle terminal, MEC server and cloud.
• HBSA [69]: HEFT-Based scheduling algorithm considers both computation and communication costs of tasks. Beginning from the exit node of the DAG task graph, it recursively determines the uplink ranking for each task by computing its average execution and communication times. Based on this ranking, the algorithm assigns each task to the resource that yields the earliest possible completion time. The start time of a task is then set according to the availability of its allocated resource and the completion times of all its predecessors.
• SchedDDQN [27]: SchedDDQN is an improved version of DQN algorithm, which aims to improve the stability and accuracy of learning by separating action selection and action evaluation. Specifically, the SchedDDQN combines the duel depth Q network and dual dqn technology, without considering the long-term dependence between tasks. The algorithm uses the behavior network to select the next action, and uses the target network to evaluate the Q value of the action.

Figure 7 benchmarks several algorithms for different numbers of subtasks. As illustrated in Figure 7a, there are significant differences in the average latency performance of different algorithms when processing varying numbers of subtasks. The observations indicate that latency generally increases for all algorithms as the number of subtasks grows. Specifically, the ASTOTL algorithm and the ASTOTC algorithm exhibit relatively high latency. This is primarily due to the limited local computing resources of the vehicle terminals and the long transmission distance to the cloud, resulting in low transmission rates for V2C communication. The RANDOS algorithm and the RRSA algorithm demonstrate similar performance. The SchedDDQN algorithm incurs high latency costs, while the DepSchedPPO algorithm and the HBSA algorithm show superior performance. The results highlight the critical importance of offloading computation to external servers as task complexity rises, ensuring that the system can maintain low latency under high workloads.

Figure 7. Performance comparison under varying subtask counts.

Figure 7b compares the energy consumption of the algorithms as the number of subtasks increases. As expected, the ASTOTC and ASTOTM algorithms show lower energy consumption due to their reliance on edge and cloud servers, which distribute the computational load. In contrast, ASTOTL exhibits an increase in energy consumption as the number of subtasks rises. The vehicle terminal must process more tasks locally, consuming more energy with the increased workload. This result underscores the significant energy advantage of offloading computations to remote servers, especially as the number of tasks grows; collectively considering Figure 7a,b, the DepSchedPPO algorithm achieves a favorable balance across both critical metrics—closely approaching the latency-optimal algorithm in delay performance while nearing the energy-optimal offloading execution algorithm in energy efficiency, highlighting its scheduling efficacy; conversely, the SchedDDQN algorithm, by neglecting long-term task dependencies, incurs prohibitively high latency costs despite reasonable energy performance, indicating its inability to effectively balance these competing objectives during learning. In Figure 7c, we observe how the Quality of Service (QoS) changes with an increasing number of subtasks. The DepSchedPPO algorithm consistently delivers superior QoS, outperforming other algorithms. It effectively balances both latency and energy consumption, ensuring high performance across varying task sizes. The ASTOTL algorithm, while effective at lower subtask counts, sees its QoS performance decline as the task load increases, highlighting the limitations of relying on local computation for larger task sets. This demonstrates the importance of considering both latency and energy in QoS optimization, as seen with the DepSchedPPO algorithm’s superior performance.

Figure 8 presents a performance comparison of the algorithms under different V2M transmission rates, where Figure 8a shows the impact of varying V2M transmission rates on latency. As transmission rates increase, latency decreases for all algorithms, but with differing patterns. The ASTOTL and ASTOTC algorithms show stable latency across different transmission rates, reflecting their reliance on external computational resources that are unaffected by transmission speed. The other algorithms, however, show more improvement with increasing V2M rates, as faster transmission allows quicker offloading to edge servers, reducing the time spent waiting for computation. In Figure 8b, the energy consumption trends are similar. The ASTOTC algorithm consistently consumes the least energy, thanks to its reliance on powerful edge and cloud servers. The ASTOTL algorithm, which depends on local computation, shows higher energy consumption, particularly at lower transmission rates, as the vehicle terminal is required to handle more tasks locally. As transmission speeds increase, the benefits of faster offloading become evident, with edge-based algorithms (especially DepSchedPPO) showing the greatest improvement in energy efficiency. Collectively analyzing Figure 8a,b, although the latency-priority algorithm excels in average latency, it exhibits relatively high energy consumption, and notably, the edge-based execution algorithm shows the most significant performance enhancement with increasing V2M rates as these rates directly impact subtask offloading speed to edge servers; crucially, the DepSchedPPO algorithm demonstrates robust performance across both latency and energy metrics under all tested V2M transmission rates.

Figure 8. Performance comparison under varying V2M transmission rates.

Figure 8c presents the performance of the algorithms on the average Quality of Service (QoS) metric under varying V2M transmission rates, showing that, except for the ASTOTL and ASTOTC algorithm, all other algorithms’ performance improves with increasing V2M rates; moreover, when the V2M rate exceeds 11 Mbps, the edge-based execution algorithm’s QoS approaches the optimal value, indicating that higher V2M rates increase the algorithm’s tendency to offload subtasks to edge servers for processing. Significantly, the DepSchedPPO algorithm consistently outperforms all others across all V2M rate levels, demonstrating its capability to effectively maximize QoS under diverse V2M conditions.

Figure 9 compares algorithm performance across different vehicle terminal computational capabilities. In Figure 9a, we observe the latency performance as the computational capabilities of the vehicle increase. The ASTOTM and ASTOTC algorithms show stable latency across all computational levels, as they rely on remote servers, whose performance is not affected by the vehicle’s computational power. The ASTOTL algorithm, however, demonstrates a noticeable reduction in latency as vehicle computational power increases. This suggests that, as the vehicle’s processing capacity grows, local execution becomes more efficient, reducing the need for offloading tasks and decreasing latency. Figure 9b compares energy consumption under varying vehicle computational capabilities. The ASTOTC algorithm continues to show the lowest energy consumption, due to its reliance on remote servers for task execution. The ASTOTL algorithm’s energy consumption decreases as the vehicle’s computational power increases, indicating that local computation becomes more energy-efficient with greater computational resources. At 3 GHz, the energy consumption of ASTOTL even becomes lower than ASTOTC, highlighting the advantages of local execution when computational resources are sufficient. Figure 9c evaluates average QoS performance across computational capabilities, where, using the local execution algorithm as the QoS reference baseline, results indicate the performance advantage of other algorithms relative to local execution diminishes—and even reverses—as terminal computational power increases, with ASTOTM and ASTOTC algorithms exhibiting negative QoS values between 2 GHz and 3 GHz, further confirming the superiority of local offloading strategies under high computational capability. Crucially, the DepSchedPPO algorithm proposed in this paper demonstrates superior performance across all tested computational capabilities. Additionally, at 3 GHz, its QoS approaches zero, indicating its increasing preference for local subtask processing on the terminal as computational capability strengthens.

Figure 9. Performance comparison under varying vehicle computational capabilities.

6. Conclusions and Future Work

In this paper, we investigate the dependency-aware computation offloading problem in vehicular edge computing environments, addressing a critical gap in existing research which predominantly assumes computationally independent tasks. To overcome this limitation, we propose a vehicle–edge–cloud-based vehicular edge computing system architecture. Our objective is to maximize system Quality of Service (QoS) for dependent tasks. To tackle the challenges posed by task dependencies, a Deep Reinforcement Learning (DRL) algorithm named DepSchedPPO (Dependency-Aware Task Scheduling based on Proximal Policy Optimization) was proposed. Specifically, the algorithm models task dependencies via Directed Acyclic Graphs (DAGs) and formulates the offloading problem as a Markov Decision Process (MDP). It employs Seq2Seq networks to approximate policy and value functions, effectively capturing long-term dependencies among DAG-structured tasks during training. A detailed explanation of the training and evaluation stages is provided, with both the Actor and Critic networks built upon Seq2Seq architectures. Subsequently, we conduct extensive simulations to validate the algorithm’s efficacy across four key metrics: value loss, policy loss, policy entropy reward, and mean reward. Experiments evaluate DepSchedPPO’s performance under varying subtask quantities, V2M transmission rates, and vehicle computational capabilities, with comparisons against baseline algorithms. Results demonstrate that DepSchedPPO efficiently makes offloading decisions in vehicular edge environments and effectively maximizes QoS.

The main contributions of this work include:

• Dependency-Aware Task Scheduling and Offloading: We introduce a novel approach that models vehicular tasks as Directed Acyclic Graphs (DAGs), ensuring that the task offloading respects both sequential constraints and parallelism, thereby improving system performance under realistic conditions.
• Deep Reinforcement Learning with PPO: We leverage the PPO algorithm to stabilize learning and enhance the offloading decision-making process, allowing for optimal scheduling in high-dimensional, dynamic environments.
• Integration of Task Dependencies and QoS Optimization: Unlike previous work, which often oversimplifies task dependencies, our approach models inter-subtask relationships and integrates Quality of Service (QoS) metrics to minimize latency and energy consumption, providing a more accurate representation of real-world vehicular applications.
• Comprehensive Evaluation: We conduct detailed evaluations under varying vehicular mobility and network conditions, showing that our method outperforms existing benchmarks, ensuring better adaptability and system robustness.

These contributions push the boundary of task offloading in VEC systems by addressing the dual challenges of dynamic task dependencies and network variability. Our method sets the stage for more efficient and reliable vehicular applications, especially those requiring real-time processing, such as autonomous driving, traffic management, and connected vehicle systems.

In the future, the proposed framework can be extended in several ways to improve its real-world applicability. First, we plan to relax the assumption that tasks are completed within a single time slot, allowing for multi-slot task execution and preemption, which would better suit real-world vehicular applications. Second, integrating advanced graph representation learning techniques, such as Graph Attention Networks (GATs) or Temporal Graph Neural Networks (TGNNs), could enhance the policy network’s ability to capture complex temporal–spatial dependencies in large-scale DAGs. Third, extending the framework to Multi-Agent Reinforcement Learning (MARL) would enable decentralized decision making across vehicles, improving scalability and robustness in ultra-dense networks. Additionally, while our current validation is based on synthetic simulations, we plan to validate the framework using real vehicular datasets and large-scale testbeds. Specifically, we will integrate the DepSchedPPO framework with Veins, SUMO, and OMNeT++ for high-fidelity simulations, capturing realistic vehicular mobility and network conditions. In the longer term, we aim to deploy the framework on a small-scale vehicular testbed to assess its feasibility under real operational conditions. Moreover, we will conduct a detailed comparison with other DRL algorithms such as A3C, SAC, and GNN-based RL. Finally, to address scalability challenges, we will explore hierarchical scheduling, task clustering, and policy distillation techniques to ensure the framework remains effective in large-scale, real-world vehicular networks.