An Enhanced MADDPG–A2C Framework for Optimized Resource Allocation in High-Speed Vehicular Networks

Abstract

To address the degradation in communication performance caused by the high mobility and dynamic uncertainty in vehicular network channels, this paper proposes a hybrid resource allocation framework that integrates the advantage actor–critic (A2C) algorithm with the multi-agent deep deterministic policy gradient (MADDPG) algorithm. By modeling the high-speed vehicular network environment, the resource allocation task is formulated as a multi-agent deep reinforcement learning (MADRL) problem within a continuous action space. The proposed framework leverages the advantage function to refine gradient estimation, thereby improving training stability and convergence behavior. Additionally, regularization penalty terms and constraint mechanisms are incorporated into the learning process to balance multiple communication objectives. Specifically, the method aims to maximize the throughput of vehicle-to-infrastructure (V2I) links while ensuring the transmission reliability of vehicle-to-vehicle (V2V) links. In simulation experiments, the proposed method performs better in terms of convergence. Compared with the conventional MADDPG algorithm, the average access success probability is improved by 1.6%, and the average V2I throughput increases by 3.5%, indicating a significant enhancement in overall vehicular communication efficiency and transmission performance.

1. Introduction

Advanced communication technologies are used by vehicular networks to propel advancements in intelligent mobility and autonomous driving [1]. The vehicular network achieves interconnectivity through vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I), and vehicle-to-pedestrian (V2P) communication technologies, thereby providing a range of intelligent transportation services that cover various aspects including road safety, traffic management, and infotainment [2,3]. In a vehicular network, resource allocation (RA) is a key component in ensuring communication efficiency and network reliability. It involves various aspects, including communication resources, vehicular resources, computational resources, and storage resources. Vehicular network resource allocation is a complex process that involves multiple layers, various strategies, and specific methods. It requires a comprehensive approach that considers the unique characteristics and requirements of the vehicular network environment, including channel dynamics, vehicle mobility, and the requirements for different quality of service (QoS) [4,5].

Traditional resource allocation methods often rely on centralized control strategies or predefined rules, which make it difficult to achieve ideal results in highly dynamic and uncertain vehicle networking environments. Conversely, reinforcement learning (RL) provides the mechanism for autonomous learning and decision-making, enabling vehicles to dynamically regulate their behavior based on changes in the environment and to find optimal resource allocation strategies. Traditional reinforcement learning, with algorithms like Q-learning and Sarsa at its core, primarily focuses on single-objective optimization for tasks such as spectrum assignment and power control in static or low-dynamic scenarios [6]. DRL addresses the dimensionality problem of traditional RL by introducing deep neural networks (DNNs), with typical algorithms including DQN (deep Q-network), DDPG (deep deterministic policy gradient), SAC (soft actor–critic), etc. MADRL is an extension and expansion of single-agent DRL in multi-agent interaction scenarios [7]. Through introducing interaction mechanisms and a distributed decision-making framework, MADRL demonstrates significant improvements compared with single-agent DRL in complex dynamic scenarios.

Despite the evolution of the resource allocation algorithm from static optimization to dynamic intelligent decision-making, which enhances complex environment adaptability, high mobility and heterogeneous requirements in vehicular networks remain critical challenges. Network resource scarcity and diverse service types further exacerbate the core dilemma in collaborative resource sharing between V2I and V2V links. Key challenges include responding to real-time network changes, increasing computational complexity (especially in multi-vehicle and multi-task scenarios); convergence instability in multi-agent algorithms within dynamic environments; and balancing fairness and efficiency via multi-objective optimization for diverse user/vehicle needs. Under this premise, reference [8] studied the issue of unfair network resource access caused by the uneven dwell time resulting from vehicle speed differences in the 5G NR V2I Mode 2 scenario. At the same time, it modeled and optimized the AoI, ensuring the timeliness of data transmission.

This paper focuses on communication resource allocation (RA) strategies for vehicular networks based on MADRL and proposes an improved joint optimization strategy for RA. The algorithm combines the MADDPG algorithm with the A2C framework to achieve efficient resource management in dynamic vehicular networks. The contributions are as follows:

(1)

Construct a high-speed dynamic model of the vehicle network environment and formulate the RA problem as an MADRL problem in a continuous action space. Establish an objective optimization problem and a reward function under multiple demand constraints, with the primary aim of maximizing V2I throughput and ensuring V2V link transmission stability, thereby fulfilling diverse QoS requirements.

(2)

Develop an optimized algorithm framework to address the convergence issues that traditional reinforcement learning algorithms encounter in high-speed dynamic environments. The algorithm combines the MADDPG algorithm with the A2C architecture. By integrating the A2C framework to optimize gradient estimation, the advantage function is embedded into the loss functions of both the actor and critic and participating in gradient calculation. This approach reduces the mean squared error and enhances the algorithm’s convergence and stability.

(3)

Design a multidimensional reward function that encompasses not only V2I and V2V link rates but also includes regularization penalty terms. Set specific constraints to ensure the reward function adapts effectively to environmental changes, enhancing learning efficiency to fulfil the requirements of real-world V2V communication environments.

The remainder of this paper is organized as follows: Section 2 reviews the existing research in this field. Section 3 introduces the system framework and formulates the optimization problem. Section 4 elaborates on the proposed resource allocation optimization scheme. Section 5 presents and discusses the simulation results. Finally, Section 6 concludes the work and highlights possible directions for future improvement.

2. Related Work

Vehicular network resource allocation algorithms have gradually transitioned from static, model-driven traditional optimization methods to dynamic, data-driven reinforcement learning (RL) and deep reinforcement learning DRL technologies, and ultimately toward intelligent multi-agent collaborative decision-making. Recent studies have improved performance and adaptability by integrating edge computing and UAV-assisted architectures, while also considering convergence and generalization capabilities in dynamic environments.

The dual-depth Q-network (DDQN) in Reference [9] is mainly applicable to single-agent environments and lacks the ability to handle multi-agent interactions. It performs poorly in continuous action spaces and collaborative or competitive tasks. In Reference [10], machine learning algorithms are combined with Kube edge computing for dynamic task offloading in RA. S. Gyawali et al. addressed the issue of collaborative misbehavior detection systems in vehicular networks failing to accurately provide feedback on attack information by proposing a dynamic reputation update approach based on DRL in Reference [11]. An improved deep reinforcement learning (IDRL) method is introduced in Reference [12] to reduce the control overhead of connected vehicle systems. Aforementioned studies considered only single-agent reinforcement learning algorithms. Although these algorithms outperform traditional methods, they struggle to achieve global optimality in complex environments, especially in non-convex optimization tasks.

Due to the inherent limitations of single-agent reinforcement learning algorithms in handling multi-agent interaction tasks, research has gradually shifted towards MADRL. MADRL, as a branch of RL, is better suited to address problems in naturally distributed and multi-participant systems such as VANETs. Among them, the MADDPG algorithm stands out as a classic approach. In reference [13], Ergün modeled resource allocation as a multi-agent deep reinforcement learning (RL) model and introduced a distributed multi-agent deep deterministic policy algorithm (MADDPG), which enhanced the capacity of the V2I link and the transmission rate of V2V links, and optimized the spectrum utilization. In reference [14], an MADDPG algorithm based on distributed execution is proposed to address the non-stationarity caused by rapid channel variations in high-speed dynamic networks. This algorithm achieves effective sharing of spectrum resources between vehicle-to-vehicle (V2V) links and vehicle-to-infrastructure (V2I) links. Although this algorithm has a relatively low computational cost, it still exhibits deficiencies in convergence performance and batch sample processing capabilities. Reference [15] proposed an improved scheme for the multi-agent coordinated actor–vritic method to address the inherent non-stationarity problem. Reference [16] introduces a conceptually simple and lightweight framework, proposing asynchronous variants of four standard reinforcement learning algorithms that have achieved success in various continuous motion control problems. Reference [17] proposed an improved MADDPG algorithm, which adopts a separation architecture design of learning and execution to solve the relevant optimization problems. Nevertheless, maintaining good convergence and model generalization in dynamic environments remains a significant challenge for this algorithm. References [18,19,20,21,22] have optimized the original multi-agent reinforcement learning algorithms to handle tasks with varying requirements. Despite the progress made by recent MADRL studies in addressing multi-agent interaction challenges, several critical limitations remain unresolved, particularly in dynamic complex environments, multi-agent coordination, and algorithmic convergence.

The integration of DRL with emerging architectures such as edge computing and UAV-assisted systems offers new opportunities for real-time and adaptive resource allocation. Edge servers can reduce processing delays by handling computation offloading near vehicles. References [23,24] leverage edge computing for both static and dynamic resource management in V2X networks. UAVs serve as mobile base stations to extend coverage in areas with limited infrastructure. References [25,26] utilize UAVs to enable reliable data communication within V2X systems. Integrating DRL with these capabilities allows for flexible resource scheduling in MADRL frameworks [27].

Achieving fast convergence, strong generalization, and low computational overhead is essential for DRL algorithms deployed in highly dynamic vehicular environments. Ahmed Manzoor and his team have proposed a secondary RA scheme using a dual-timescale approach to efficiently allocate idle resources among stationary and mobile vehicles. By integrating this mechanism with the QMIX algorithm, they have enhanced computational resource allocation [28]. The PPO algorithm, introduced in 2017, achieved a breakthrough in training stability through clipping of the objective function. However, it has limitations in multi-agent scenarios, including high sample complexity, slow convergence speed, and insufficient scalability. Particularly in large-scale vehicular networks, maintaining the trust region of multi-agent interactions leads to exponential growth in computational overhead [29].

M. Babaeizadeh et al. have proposed an A2C algorithm framework that, through parallel processing, supports multiple agents operating in different environmental instances simultaneously [30]. A2C addresses the high variance issue associated with the basic actor–critic algorithm by introducing a baseline to the calculation of expected rewards. Compared with traditional reinforcement learning algorithms, the A2C algorithm can effectively reduce the data correlation and update instability by using multiple learners. The on-policy characteristic of the A2C algorithm enables an immediate link between sample collection and policy updates, meeting the millisecond-level response requirements of vehicles. Compared to PPO and SAC, the A2C algorithm exhibits superior computational efficiency due to its concise architecture and synchronous update mechanism, which reduces latency in real-time decision-making and is well suited to vehicle system resource constraints. The MAPPO mainly utilizes the advantage function to clip the objective function to restrict policy updates. Although this ensures stability, it may overly restrict policy exploration, leading to slow convergence in complex dynamic environments. In terms of environmental stability, A2C optimizes reward bias through the advantage function, mitigating training fluctuations caused by vehicle dynamic topology, thereby overcoming the significant sampling bias of PPO and the excessive exploration characteristics of SAC. A2C has been extensively validated in vehicle network optimization, facilitating comparison with existing research baselines and ensuring the reliability of conclusions. However, it may also lead to low sample efficiency. As the number of vehicles rises, it will significantly reduce the convergence speed of the training process [31].

Furthermore, data compression technology is crucial for enhancing the efficiency of vehicle-to-infrastructure (V2I) and vehicle-to-vehicle (V2V) communication. For instance, when confronted with the challenges of limited bandwidth and high data transmission demands, Y. Wiseman [32] proposed a two-step reduction method for the transmitted images, significantly reducing the data transmission volume required for the vehicle networking system.

Table 1. Literature comparison.

Algorithm	Reference	Limitations	Environment	Convergence	Scalability
DQN	[6], 2024; [9], 2023; [12], 2023	Hard to handle continuous actions; unstable in high-dynamic vehicular networks	Single-agent; discrete spectrum allocation	Slow; unstable in complex scenarios	Limited scalability
DDPG	[5], 2023; [18], 2023; [23], 2020	Sensitive to hyperparameters; overestimation bias	Continuous power control; single-agent	Better than DQN but unstable under dynamic channels	Limited scalability to multi-agent
PPO	[4], 2023; [24], 2023; [25], 2024	High sampling cost; limited fairness; weak generalization in large-scale IoV	Single-agent; continuous and discrete hybrid resource allocation	Stable convergence, better than DDPG/DQN	Poor scalability to multi-agent
MAPPO	[2], 2025; [29], 2025; [33], 2024	High computational cost for centralized critic; bottleneck in large networks	Multi-agent; CTDE framework	Stable; faster convergence than MADDPG	Better than PPO but still limited in very large-scale IoV
MADDPG	[13], 2025 [15], 2017; [19], 2022; [20], 2020	Training instability due to non-stationarity; high variance	Multi-agent; CTDE with deterministic policies	Converges faster than DDPG but is less stable than MAPPO	Limited scalability when the number of agents increases
MADQN	[9], 2023; [17], 2022; [34], 2025	Limited to discrete action spaces; poor adaptability to mixed continuous/discrete tasks	Multi-agent; value-based	Converges more slowly than policy gradient methods	Moderate scalability but struggles in continuous IoV tasks
A2C	[16], 2016; [30], 2024; [31], 2024	Sample inefficiency; sensitive to learning rates	Single-agent; synchronous actor–critic	Faster convergence than vanilla policy gradient; less stable than PPO	Limited scalability; mostly used in small IoV environments

is a comparison of the literature.

3. System Model and Problem Formulation

3.1. System Model

This research primarily investigates the vehicular communication environment under high-speed dynamic networks, considering the scenario depicted in Figure 1, which includes a base station and multiple vehicular users. These vehicles are categorized as V2I users and V2V users depending on their communication requirements. The V2I link enables communication between base stations, vehicles, and roadside units, while the V2V links facilitate direct communication between the current vehicle and neighboring vehicles. The set of V2I vehicles links is denoted as S_V2I = {1, 2, …, M}, and the set of V2V vehicles links is denoted as S_V2V = {1, 2, …, N}. To conserve spectral resources, multiple V2V users can reuse the spectrum resources of V2I links to achieve sharing.

3.2. Communication Model

In the vehicular network environment constructed in this paper, it is presumed that the fading characteristics of each orthogonal sub-channel are identical and independent of each other. The total spectral resources of this uplink channel are evenly divided among M orthogonal sub-channels. To allocate the V2I link spectral resources more reasonably and efficiently, thereby fully utilizing the channel conditions, this paper employs a fully distributed resource allocation mechanism. It is assumed that all vehicular user transceivers employ a single antenna, and that each V2I vehicular user has exactly one V2I uplink channel for communication. The communication model considers slow fading caused by building blockage and changes in path length, as well as fast fading caused by multipath propagation, and both phenomena coexist.

3.2.1. Interference Analysis

In the vehicular networking communication system, various interference sources will have a serious impact on the communication quality. On sub-channel $m$, the interference from the n-th V2V link is denoted by $I_n^v\left[m\right]$, and from the rest of the V2V links, it is represented by $I_k^v\left[m\right]$:

$$\begin{array}{c} I_n^v\left[m\right]={\displaystyle \sum _{n=1}^N}{\delta }_n\left[m\right]p_n^v\left[m\right]g_{n,B}[m]\end{array}$$

(1)

$$\begin{array}{c} I_k^v\left[m\right]={\displaystyle \sum _{k\ne n}^N}{\delta }_k\left[m\right]p_k^v\left[m\right]g_{k,n}\left[m\right]\end{array}$$

(2)

where $p_n^v\left[m\right]$ represents the transmission power of the n-th V2V link, $g_{n,B}[m]$ denotes the channel gain from the V2V link transmitter to the base station, ${\delta }_n\left[m\right]$ indicates whether the m-th V2I link is reused by the n-th V2V link, with a value of 1 if it is reused and 0 otherwise, and $g_{k,n}\left[m\right]$ represents the channel gain of the k-th V2V link.

The interference by the m-th V2I link is denoted by $I_m^I[m]$:

$$\begin{array}{c} I_m^I\left[m\right]=P_m^I\left[m\right]g_{m,n}\left[m\right]\end{array}$$

(3)

The SINR for the m-th V2I link and the n-th V2V link are, respectively, expressed as

$$\begin{array}{c} {\gamma }_m^I\left[m\right]={\displaystyle \frac{p_m^I\left[m\right]g_{m,B}[m]}{I_n^v\left[m\right]+{\sigma }^2}}\end{array}$$

(4)

$$\begin{array}{c} {\gamma }_n^v\left[m\right]={\displaystyle \frac{p_n^v\left[m\right]g_n\left[m\right]}{I_m^I[m]+I_k^v\left[m\right]{+\sigma }^2}}\end{array}$$

(5)

where ${\sigma }^2$ indicates the power of Gaussian white noise, and $g_n\left[m\right]$ signifies the channel gain of the n-th vehicle on the m-th orthogonal sub-channel.

3.2.2. Throughput Analysis

According to Shannon’s formula [33], the total throughput of the V2I and V2V is derived, respectively, as $R_m^I\left[m\right]$ and $R_n^v\left[m\right]$:

$$\begin{array}{c}{ R}_m^I\left[m\right]=Blb\left(1+{\gamma }_m^I[m]\right)\end{array}$$

(6)

$$\begin{array}{c} R_n^v\left[m\right]=Blb\left(1+{\gamma }_n^v\left[m\right]\right)\end{array}$$

(7)

where $B$ represents the channel bandwidth.

Since the travel speed and traffic position status of V2V communication users are key to service safety information, ensuring the effective transmission probability of V2V is crucial. The effective transmission probability ${\psi }_n$ is described as follows:

$$\begin{array}{c} {\psi }_n=p_r\left\{{\displaystyle \sum _{t=1}^T}{\displaystyle \sum _{m=1}^M}{\delta }_n\left[m\right]R_n^v\left[m,t\right]\ge {\displaystyle \frac{Bp}{△T}}\right\},n\in N\end{array}$$

(8)

Among them, it should be noted that $Bp$ indicates the size of the generated V2V payload, $t$ represents the time slot, and $△T$ denotes the coherence time.

3.3. Problem Formulation

The purpose of this paper is to improve the overall throughput of V2I links while ensuring the transmission reliability of V2V links, focusing on the average transmission success probability of V2V links. By considering the key quality of service constraints such as delay and channel interference, the following reliability assurance optimization problems are formulated:

$$\begin{array}{c}OP1:{max}_{\left(p_n^v\left[m\right],p_m^I[m],{\delta }_n\left[m\right]\right)}{\displaystyle \sum _{m=1}^M}{R}_m^I[m]\end{array}$$

(9)

$$\mathit{OP}2:\underset{\left(p_n^v[m],p_m^I[m],{\delta }_n[m]\right)}{\mathit{max}}{\mathbb{E}}_{\mathit{h}}\left[1-{\displaystyle \frac{n\prime }{N}}\right]$$

(10)

where $n^{\prime }$ denotes the number of V2V links that fail to satisfy the minimum payload transmission requirement within the coherence time $△T$, defined as

$$n\prime =\sum _{n=1}^N\mathbb{I}\left(\sum _{t=1}^T\sum _{m=1}^M{\delta }_n[m]R_n^v[m,t]<{\displaystyle \frac{B_p}{\mathsf{\Delta }T}}\right)$$

(11)

In Equation (11), $\mathbb{I}(\cdot )$ represents the indicator function that equals 1 when the condition is satisfied (i.e., the n-th V2V link fails to transmit the required payload Bp within $△T$) and 0 otherwise. ${\mathbb{E}}_{\mathbf{h}}[\cdot ]$ denotes the expectation over all possible channel states.

Remark 1:

The objective function in OP2 is mathematically equivalent to maximizing the average access success probability of V2V links. Specifically,

$${\mathbb{E}}_h\left[1-{\displaystyle \frac{n\prime }{N}}\right]={\displaystyle \frac{1}{N}}{\sum _{n=1}^N\mathbb{E}}_h\left[\mathbb{I}\left(\sum _{t=1}^T\sum _{m=1}^M{\delta }_n[m]R_n^v[m,t]\ge {\displaystyle \frac{B_p}{\Delta T}}\right)\right]={\displaystyle \frac{1}{N}}\sum _{n=1}^N{\psi }_{n }$$

(12)

where ${\psi }_n$ is the effective transmission probability of the n-th V2V link as defined in Equation (8). Therefore, OP2 directly corresponds to constraint C4 (${\psi }_n\ge {\rho }_0$), ensuring the transmission reliability of V2V links.

• $s.t.$

$$C1:0\le p_m^I[m]\le p_{max};0\le p_n^v\left[m\right]\le p_{max};\forall m\in M,\forall n\in N$$

(13)

$$\begin{array}{c} C2:0\le I_n^v\left[m\right]\le I_{max};0\le I_m^I[m]\le I_{max};0\le I_k^v\left[m\right]\le I_{max}; \forall m\in M,\forall n\in N\end{array}$$

(14)

$$\begin{array}{c} C3:R_m^I[m]\ge R_{min}; \forall m\in M\end{array}$$

(15)

$$\begin{array}{c} C4:{\psi }_n\ge {\rho }_0; \forall m\in M,\forall n\in N\end{array}$$

(16)

$$\begin{array}{c} C5:{\tau }_n^v\le △T; \forall n\in N\end{array}$$

(17)

$$\begin{array}{c} C6:{\delta }_n\left[m\right]\in \left\{0,1\right\}; \forall m\in M,\forall n\in N\end{array}$$

(18)

$$\begin{array}{c} C7:R_n^v\left[m\right]\le R_{max};\forall n\in N \end{array}$$

(19)

where $N$ represents the total number of V2V communication links. C1 represents the maximum permitted transmission power. C2 represents the limitations of interference at the receiving end of the links. C3 represents the minimum throughput constraint that the V2I link can tolerate. C4 represents the effective transmission probability constraint, which is consistent with the objective of OP2. C5 represents the maximum transmission delay allowed. C6 represents whether the spectrum resources of the V2I link have been reused, which is a binary variable. C7 represents the maximum throughput constraint that the V2V link can tolerate, that is, the maximum transmission rate constraint.

4. Proposed Resource Allocation Optimization Strategy Based on MADDPG and A2C

Building upon an in-depth study of the MADDPG and A2C algorithms, an improved MADDPG-A2C joint optimization algorithm for resource allocation has been proposed in this paper. The proposed strategy adopts the centralized training with decentralized execution (CTDE) framework of MADDPG to address multi-agent collaborative decision-making problems. It introduces the advantage function mechanism of A2C to optimize the gradient estimation process. By incorporating the advantage function from A2C into the actor and critic loss functions of MADDPG through multiplicative weighting, it enables direct participation in gradient calculations, thereby addressing issues of unstable training and large gradient variance in traditional MADDPG. The combination of the MADDPG and A2C algorithms provides an effective learning framework for multi-agent systems, enabling better strategy learning and resource allocation under dynamic and complex environmental conditions.

4.1. State Space Design

In time slot $t$, the environmental state that agent $n$ currently encounters is denoted as $s_t^n$. Agent $n$ then produces the corresponding action $A_t^n$ according to the current policy π. For the vehicular network as a whole, the joint state space formed by all agents is denoted by $S_t$, and the joint action $A_t$ is generated under the same policy π, resulting in a joint reward $R_t$ after executing the actions together. In the neural network, when the agent $n$ of the state $s_t^n$ transits to the next state, it is denoted as $s_{t+1}^n,$ with probability ${\rho }_t^n$. For each independent agent, taking agent $n$ as an example, it can only obtain partial state information and the action space of its own environment, but cannot obtain global state information and the action space information of other agents, which are all unknown factors. The expression for ${\rho }_t^n$ is as follows:

$$\begin{array}{c} {\rho }_t^n=p_r\left({\displaystyle \frac{s_t^n}{s_{t+1}^n}},A_t^n\right)\end{array}$$

(20)

The expression for $s_t^n$ is

$$\begin{array}{c} s_t^n=\left(g_n\left[m\right],g_{k,n}\left[m\right],g_{n,B}\left[m\right],g_{m,n}\left[m\right]\right)\end{array}$$

(21)

The state space inherently includes dynamic information such as vehicle position and speed through real-time updated channel gain parameters. At the beginning of each time slot, the system first calculates and updates the path loss and small-scale fading parameters of all links according to the real-time position coordinates and moving speed of vehicles, and then uses the updated channel gains as the state input, ensuring that the state representation can fully reflect the time-varying characteristics of high-speed dynamic networks.

4.2. Action Space Design

The vehicular network resource allocation problem primarily concerns the optimization of orthogonal sub-channel allocation for V2V communication and transmission power. The spectrum resources of each V2I link are evenly divided into M orthogonal sub-channels. The n-th V2V link can select one channel from these M orthogonal sub-channels for sharing and adjust their transmission power. The boundaries of the continuous power value $p_n^v\left[m\right]$ are defined as 0 ≤ $p_n^v\left[m\right]$ ≤ $P_{max}$. The action selected by agent $n$ is given by

$$\begin{array}{c} A_t^n=\left\{p_n^v\left[m\right]\right\}\end{array}$$

(22)

4.3. Reward Function Design

The optimization problem is established in a multi-objective optimization framework and is converted into a single-objective problem through a weighted linear combination. In a unit time slot $t$, the objective function treats completed transmissions and unfinished transmissions as two separate cases. The first objective function is the reward for the V2I link, defined as the total throughput of the V2I link. For V2V users, at time $t$, the reward related to unfinished V2V transmissions is defined according to the payload that has been delivered. The second objective function encourages V2V users to increase the transmission rate by assigning a scalar value that is larger than the effective V2V transmission rate. The two objective functions can be written as follows:

$$\begin{array}{c} {Re}_{V2I}={\sum }_{m=1}^M{R}_m^I[m]\end{array}$$

(23)

$$\begin{array}{c} {Re}_{V2V}=\left\{\begin{array}{c}{\sum }_{m=1}^M{\delta }_n\left[m\right]R_n^v\left[m,t\right],B\ge 0\\ \Phi ,B<0\end{array}\right. \end{array}$$

(24)

Then, by applying weight coefficients, the reward function is obtained as follows:

$$\begin{array}{c} {Re}_{total}=\lambda \ast {Re}_{V2I}+\left(1-\lambda \right)\ast {\sum }_{n=1}^N{Re}_{V2V}-\phi \end{array}$$

(25)

In Equation (25), $\lambda$ represents the weight coefficient, and it is within the range (0, 1), indicating the importance level of this element’s reward. $\phi$ is the penalty factor, which is 0 when certain constraint conditions are met and $\zeta$ when the constraints are not met. The expression for $\zeta$ is as follows:

$$\begin{array}{c} \zeta =\chi \ast {\displaystyle \frac{{Re}_{V2V}}{\eta }}\end{array}$$

(26)

In Equation (26), $\chi$ is an adjustable parameter used to control the degree of the penalty. $\eta$ represents the logarithm of the number of communication links.

To maintain the stability of rewards while accounting for current rewards and expected future rewards, this algorithm aims to determine the optimal strategy $\pi \ast$ for maximizing the cumulative expected reward; that is,

$$\begin{array}{c} {Re}_{max}=E_{\pi }\left[{\sum }_{\tau =0}^{\infty }{\varpi }^{\tau }]{Re}_{t+\tau }\right];0\le \varpi \le 1\end{array}$$

(27)

4.4. Joint Optimization Algorithm Based on MADDPG and A2C

Aiming to combat the inevitable multi-agent concurrent behaviors in the traditional MADDPG algorithm, which lead to the drawbacks of low sample efficiency and unstable training, an improved cooperative optimization architecture integrating MADDPG and A2C is proposed, as shown in Figure 2. Figure 2 illustrates the system architecture and the reinforcement learning operation process. Specifically, the left side of the figure presents the V2I/V2V hierarchical communication scenario, while the right side shows the operational flow of the MADDPG-A2C multi-agent learning framework. Therefore, this diagram comprehensively presents the core design of this study from two dimensions: the communication layer and the reinforcement learning framework operation process.

To this end, traditional MADDPG integrates the A2C framework, leveraging the classical advantage function to compute policy gradients while updating the loss functions of both actor and critic networks. Accordingly, this approach minimizes mean squared error, simplifies handling of state spaces and action spaces, and effectively boosts system sampling efficiency and performance stability.

For clarity, the overall framework of the optimized algorithm and the system process are shown in Figure 3. The algorithm in this study adopts centralized training and distributed execution. Each independent agent $n$, where n ∈ (1, 2, …, N), outputs the corresponding action ${\mathrm{A}}_{\mathrm{t}}^{\mathrm{n}}$ through the actor network according to its own state $s_t^n$. There are $N$ independent actor networks for $N$ agents, corresponding to $N$ policy functions and $N$ critic networks. The policy function of the actor network is denoted as ${\pi }_n$ policy. For a given critic, the input contains the outputs of $N$ actor networks and finally provides feedback, with the feedback value denoted as $Q_n$. As training proceeds over multiple iterations, the global optimal solution is eventually derived, maximizing the reward return and causing the reward value to gradually converge. In summary, the improved MADDPG-A2C algorithm introduces the classic advantage function of the A2C algorithm into the original MADDPG framework to reduce mean square error and enhance efficiency and stability.

In the MADDPG-A2C framework, two typical network architectures are used, namely, the actor and critic. The actor network is used to collect the corresponding environmental states and then execute the corresponding actions according to policy $\pi$, while the critic network uses the state–action value function to assess the quality of action $A_t^n$ according to the policy. The current environmental state of agent $n$ is $s_t^n$, and $\gamma$ is the discount factor of rewards, which ranges from 0 to 1. In implementation, the continuous action variable space is discretized into four normalized levels, i.e., [−10, 5, 15, 23] dBm [34,35]. Therefore, the state–action value function of MADDPG-A2C is as follows:

$$\begin{array}{c}{ Q}_n\left(s_t^n,A_t^n\right)=E\left[R_t^n+\gamma Q\left(s_{t+1}^n,A_{t+1}^n\right)\right]\end{array}$$

(28)

The optimal action plan is derived under the optimal strategy ${\pi }_{\mathrm{n}}$ to maximize the expected return. The expressions are as follows:

$$\begin{array}{c} Acto{r}_{Loss}=-E\left[Q_n\left(s_t^n,A_t^n|\omega \right)\ast advantages\right]\end{array}$$

(29)

$$\begin{array}{c} {Critic}_{Loss}=E\left\{{\left[\left(y-Q_n\left(s_t^n,A_t^1,\dots \dots ,A_t^N\right)\right)\ast advantages\right]}^2\right\}\end{array}$$

(30)

where y represents the target Q-value, and the advantage function is given by

$$\begin{array}{c} A\left(s_t^n,A_t^n\right)=Q\left(s_t^n,A_t^n\right)-V\left(s_t^n\right)\end{array}$$

(31)

The essence of the advantage function $A\left(s_t^n,A_t^n\right)=Q\left(s_t^n,A_t^n\right)-V\left(s_t^n\right)$ is to eliminate the state value function $V\left(s_t^n\right)$ as a baseline, thereby reducing the variance in the policy gradient estimation. In the standard policy gradient theorem, the policy gradient can be expressed as

$$\mathsf{\nabla }J\left(\theta \right)=E[\mathsf{\nabla }log\pi \theta (s_t^n\left|A_t^n\right) \ast Q\left(s_t^n,A_t^n\right)]$$

(32)

After introducing the advantage function, the policy gradient becomes

$$\mathsf{\nabla }J\left(\theta \right)=E[\mathsf{\nabla }log\pi \theta (s_t^n\left|A_t^n\right) \ast A\left(s_t^n,A_t^n\right)]$$

(33)

Here, $A\left(s_t^n,A_t^n\right)$ = $Q\left(s_t^n,A_t^n\right)$ − $V\left(s_t^n\right)$. Since E [$A\left(s_t^n,A_t^n\right)$)$|s_t^n$] = 0, the advantage function maintains the unbiasedness of the gradient and significantly reduces the variance, as Var[$A\left(s_t^n,A_t^n\right)$] ≤ Var[$Q\left(s_t^n,A_t^n\right)$].

A lower-variance gradient leads to more stable learning dynamics with fewer oscillations, while the centered signal $A\left(s_t^n,A_t^n\right)=Q\left(s_t^n,A_t^n\right)-V\left(s_t^n\right)$ emphasizes the relative action quality rather than state difficulty. As a result, the actor can make more consistent progress and achieve faster practical convergence under the same training budget and is less likely to take noisy steps that cause oscillations or temporary performance drops; consequently, the learning curve becomes more stable and empirically closer to a monotonic improvement trend.

When the ${Critic}_{Loss}$ is continuously differentiable, the parameters $\omega$ can be updated through gradient descent. Here, the update strategy of the A2C algorithm is adopted as

$$\begin{array}{c} {\omega }_n\leftarrow {\omega }_n+{\beta }_{critic}\mathsf{\nabla }{V}_{{\omega }_n}\left(s_t^n\right)\ast \epsilon \end{array}$$

(34)

where ${\beta }_{critic}$ represents the learning rate and $\epsilon$ is the Temporal Difference (TD) error. Each agent modifies the evaluation network parameters of the actor by maximizing the objective function when it seeks to maximize the cumulative reward. The objective function is given by

$$\begin{array}{c} J_{\theta }=E\left[Q_n\left(s_t^n,A_t^n\right)\right]\end{array}$$

(35)

Similarly, $J_{\theta }$ is continuously differentiable, allowing for gradient updates to the parameters $\theta$. The update strategy of the A2C algorithm is adopted as

$$\begin{array}{c}{ \theta }_n\leftarrow {\theta }_n+{\alpha }_{actor}{\mathsf{\nabla }}_{{\theta }_n}log{\pi }_{{\theta }_n}\left(s_t^n|A_t^n\right)\ast \epsilon \end{array}$$

(36)

where ${\alpha }_{actor}$ is the learning rate of the actor network. $\epsilon$ is TD error. The expression for the soft update is as follows:

$$\begin{array}{c} {\theta }_n^{\prime }\leftarrow \tau \ast {\theta }_n+\left(1-\tau \right)\ast {\theta }_n^{\prime }\end{array}$$

(37)

$$\begin{array}{c} {\omega }_n^{\prime }\leftarrow \tau \ast {\omega }_n+\left(1-\tau \right)\ast {\omega }_n^{\prime }\end{array}$$

(38)

The agent generates actions through an independent actor network. The parameters of the critic network and actor network adopt the synchronous update strategy of A2C, combined with the Temporal Difference (TD) error to achieve efficient parameter iteration. The target network still follows the soft update mechanism of MADDPG to ensure training stability. After being centrally evaluated and fed back by their respective critic networks, the networks are updated and iteratively trained in combination with the A2C advantage function to converge to the global optimal state. Algorithm 1 shows the pseudocode for the proposed MADDPG-A2C algorithm, and Figure 4 shows the flowchart of the algorithm.

In the MADDPG framework, the critic network needs to handle the global state and all agents’ actions, and its input dimension grows linearly with the number of agents N. The advantage function calculation introduced by MADDPG-A2C involves additional state value estimation, but this operation is of the same order as the forward propagation of the critic network. Therefore, the asymptotic complexity of MADDPG-A2C is at the same level as that of MADDPG. MADDPG-A2C adds the estimation of the state value function V(s) on top of MADDPG. The structure of the V network is of the same order as the critic, with a computational complexity of O(Nd·d) = O(Nd²), and the calculation of the advantage function A = Q − V is an element-wise operation of O (1). Therefore, the total complexity remains O(N·d²), where N is the number of agents and d is the dimension of the state space (assuming that the state dimension d_s, action dimension d_a, and hidden-layer dimension $h$ are of the same order).

The proposed MADDPG-A2C algorithm is described in Algorithm 1.

Algorithm 1. Proposed MADDPG-A2C Algorithm.

The proposed framework is built upon the MADDPG architecture. The advantage function is used to refine the gradient estimation and reduce the mean squared error. It is directly embedded into the loss functions of the actor and critic through multiplicative weighting. This design enables the advantage function to directly participate in the gradient calculation, thereby improving the estimation accuracy. At the same time, the advantage function in the framework can directly correct the Q-value gradient to quickly adapt to the changes in signal-to-noise ratio caused by high-speed movement. Moreover, the proposed framework uses the advantage function to weight multidimensional rewards, ensuring the reliability of V2V transmission while maximizing V2I throughput, avoiding the inherent over-conservative problem of the MAPPO clipping mechanism.

In the proposed centralized training and decentralized execution framework, the global observations required by the centralized critic are aggregated at the edge side by an RSU or a BS. The aggregated information includes channel and interference measurements, resource occupancy, and link-level QoS states such as remaining payload and remaining deadline. These observations are used to train the centralized critic together with the joint actions stored in the replay buffer. During online execution, each vehicle runs only the decentralized actor and makes decisions based on local observations. Therefore, per-slot, low-latency decisions do not rely on real-time global information exchange, while the critic-related aggregation is confined to the training phase and can be performed at a slower timescale.

5. Experimental Analysis and Discussion

5.1. Simulation Environment Setup

An urban vehicular network simulator based on 3GPP TR 37.885 [35] was implemented in Python 3.7 (TensorFlow 1.14.0) and run on a single machine equipped with an NVIDIA RTX 4060 (8 GB VRAM). Bandwidth was set to 4 MHz based on 3GPP V2X side link recommendations (1.4–20 MHz) and literature [35,36], balancing multi-agent communication needs and computational complexity. The Adam optimizer with learning rate 0.001 was selected via grid search over {3 × 10⁻⁴, 10⁻³, 3 × 10⁻³} to balance convergence speed and stability in multi-agent settings, leveraging deep reinforcement learning actor–critic tuning experience. Assuming a one-to-one correspondence between V2V and V2I links, different sizes of V2V link loads are used to confirm the robustness of the algorithm.

Table 2. Simulation parameters.

Vehicular Environment Parameters	Value
Carrier frequency $f_c$	2 GHz
Number of V2I links $M$	4
Number of V2V links $N$	4
Bandwidth $B$	4 MHz
Noise power $N_0$	−114 dBm
Base station antenna height	25 m
Base station single-antenna gain $g$	8 dBi
Base station noise figure	5 dB
Vehicle noise figure	9 dB
Vehicle speed $V$	5–50m/s
Time constraint	100 ms
Max transmission power of the V2I $P_{max}^{V2I}$	0.199 w
Max transmission power of the V2V $P_{max}^{V2V}$	0.199 w
Transmission power of the V2I $P^{V2I}$	[−10, 5, 15, 23] dBm
Vehicle antenna height	1.5 m
V2V link path loss model	$22.7\lg \left(d\right)+41+20\lg \left(f_c/5\right)$
V2I link path loss model	$128.1+37.6\lg \left(d\right)$
Fast-fading update	1 ms

presents the simulation parameters.

The maximum V2V transmission power is divided into specific levels to better control interference and facilitate quantization during the algorithm implementation and simulation, and to reduce the computational complexity. V2I link models mainly focus on the impact of distance. V2V communication is usually carried out in a higher frequency range (such as on-board Wi-Fi, LTE, etc.), and the model should consider not only distance but also the impact of frequency on path loss. Therefore, the $128.1+37.6\lg \left(d\right)$ model is selected for V2I links, and the $22.7\lg \left(d\right)+41+20\lg \left(f_c/5\right)$ model is selected for V2V links. Based on actual application scenarios, the vehicle speed V was set to 5–50 m/s, and the V2I transmission power and path loss were modeled as dynamic variables.

5.2. Ablation Experiment

We verify the contributions of three reinforcement learning algorithms (DDPG, A2C, and MADDPG) and the improved reward function (Reward*) by comparing the performance metrics of various component combinations.

Table 3. Comparison of ablation experiment results.

Index	Methods				Reward	V2V Links’ Success Probability	V2I Links’ Throughput (Mbps)	MFLOPS (M)
	DDPG	A2C	MADDPG	Reward*
1	√				64.47	0.688	27.43	0.103
2		√			85.97	0.8217	28.61	0.108
3			√		87.59	0.837	29.8	0.43
4		√	√		87.72	0.84	29.75	0.516
5		√	√	√	88.75	0.851	30.84	0.516

Reward* denotes the improved reward function, √ indicates that this module has been used and bold black indicates the best performance metric.

presents the performance metrics (reward, V2V success probability, V2I throughput, and MFLOPS) of five experimental groups with different method combinations.

The ablation experiment demonstrates that the MADDPG-A2C strategy proposed in this paper, which is a combination of A2C, MADDPG, and an improved reward function, is the optimal configuration. It achieves the highest reward value (88.75), vehicle-to-vehicle communication success probability (0.851), and vehicle-to-infrastructure communication throughput (30.84 Mbps). This confirms that multi-agent collaboration (MADDPG) and single-agent optimization (A2C) have a synergistic effect and that the improved reward function effectively enhances system performance without increasing additional computational costs. Single-algorithm groups (DDPG and A2C) have lower MFLOPS but poorer performance, while multi-algorithm combinations improve performance at the cost of higher computational load.

5.3. Comparative Experimental Analysis

The comparative experiment mainly compares different algorithms in terms of reward, average throughput, average access success probability, and total throughput. The reward comparison under different algorithms is depicted in Figure 5. Semi-transparent shading is used to represent the 95% confidence interval (CI) ranges. The figure illustrates the gradual convergence of reward values for different algorithms with the increase in training rounds. The MADDPG-A2C algorithm proposed in this paper converges the fastest. Compared with other algorithms, MADDPG-A2C has the narrowest 95% confidence interval, indicating a smaller sampling error in its mean estimation and higher reliability. This is because the MADDPG-A2C algorithm proposed in this paper introduces an appropriate advantage function, which makes the reward feedback starting point higher, and at the same time, it can become stable and converge faster under fewer training episodes.

Figure 6 illustrates the impact of the number of training rounds under different algorithms on the average V2I link throughput. The shaded area in the figure represents the 95% confidence interval. When the training set size is small, the average V2I throughput obtained by each algorithm fluctuates significantly, and the confidence interval (CI) width is relatively wide. As the size of the training set increases gradually, the average throughput shows an upward trend and eventually converges, with the 95% confidence interval narrowing. Among them, the MADQN algorithm converges the fastest. Simultaneously, it is evident that under the same number of training sets, the proposed MADDPG-A2C algorithm can obtain a higher average V2I link throughput. This is mainly due to the fact that the neural network in the MADQN algorithm contains only one Q-network, and the DDPG algorithm adopts single-agent training and learning, which cannot obtain the global optimal solution, so the rate is low. The RANDOM algorithm can avoid corresponding errors when exploring unknown environments and help to avoid the model falling into local optimal solutions. Theoretically, its performance is better than that of MADQN and DDPG. However, the RANDOM algorithm is generally suitable for dealing with simple environments.

Figure 7 shows the change in the average access success probability of V2V links under different training episodes. Among the five algorithms, the proposed MADDPG-A2C algorithm maintains a relatively high success probability at around 0.85 under different training episodes, which is relatively stable, and its access success probability does not fluctuate significantly with the change in the number of episodes. Among the five algorithms, the DDPG algorithm uses a single agent, so the success probability is relatively low, and the fluctuation is obvious. Figure 7 illustrates that the proposed algorithm has excellent performance.

Figure 8 reflects the relationship between V2V load size and the average success probability of V2V transmission. The success probabilities for all five algorithms decrease as the V2V load size increases, with the proposed algorithm showing the slowest decline. To exclude minimal probability effects, a load peak of 6 × 1060 Bytes is considered. At a load size of 1 × 1060 Bytes, all five schemes exhibit high success probabilities, with the proposed algorithm achieving the highest. At a load size of 3 × 1060 Bytes, the proposed algorithm outperforms MADDPG, MADQN, RANDOM, and DDPG by 18.1%, 85.7%, 54.7%, and 47.7%, respectively. Experimental results demonstrate that the MADDPG-A2C algorithm consistently maintains the highest average success probability across varying V2V load sizes.

Figure 9 shows the relationship between the variation in V2V load size under various methods and the total V2I throughput. In Figure 9, vertical error bars are added at each load size point. The vertical error bars in the figure represent ±1 standard deviation (SD) of the data. As shown in the figure, the total V2I throughput decreases when the load size increases due to load increases and effective communication resource utilization decreases, resulting in a decrease in system throughput. However, the advantage function is introduced into the algorithm structure proposed in the paper, which alleviates the negative correlation between load size and throughput. The shortest error bars of MADDPG-A2C indicate optimal performance stability. When the load size is 3 × 1060 bytes, the MADDPD-A2C algorithm achieves a performance improvement of 12.5%, 58.1%, 22.7%, and 28.5% over the MADDPG, MADQN, RANDOM, and DDPG baseline algorithms, respectively. Therefore, the total throughput of V2I decreases slightly with the increase in load size of V2V, which has obvious advantages among different algorithms.

Figure 10 shows the transfer rate for the four V2V links under the MADDPG, MADQN, RANDOM, DDPG, and MADDPG-A2C algorithms, respectively. This simulation selects four vehicles (i.e., four vehicle-to-vehicle links) with a load size of 2 × 1060 bytes. It compares the instantaneous rate variations of all V2V links across the five algorithms within 30 ms. It can be observed that all loads of the MADDPG-A2C and MADDPG were successfully transmitted within 17 ms. Compared with the MADDPG algorithm, the instantaneous rate changes of the four links in the MADDPG-A2C algorithm are more stable. The RANDOM algorithm successfully transmitted all link loads within 25 ms. In the MADQN algorithm, only Link 2 was successfully transmitted within 24 ms, while the other links were not completed. In the DDPG algorithm, only Link 3 and Link 4 completed the transmission tasks. This indicates that the MADDPG-A2C algorithm exhibits high efficiency and stability in multi-agent collaborative learning.

Figure 11 shows the curves of the transfer rate variation with step for four V2V links (Link 1–Link 4) under different algorithms. When the V2V link rate drops to 0, it indicates that the load transmission and delivery task is completed. In V2V Link 1, MADDPG-A2C and MADDPG completed the load transmission task in 17 ms and 16 ms, respectively, which is the shortest time required among the five algorithms. Compared to MADDPG, MADDPG-A2C has a higher average rate and greater stability. In V2V Link 2, Link 3, and Link 4, for the same V2V load transmission task, the MADDPG-A2C algorithm also takes the shortest time, followed by the MADDPG algorithm. This indicates that the agents in the MADDPG-A2C scheme can complete the load transmission task more quickly, and there are no significant fluctuations in each link when completing its own transmission tasks, thereby reducing interference to other links.

To evaluate the robustness of the proposed method under different traffic densities and mobility levels, we conduct additional experiments by varying the number of agents and the vehicle speed range. Specifically, we consider three density settings with 4, 8, and 12 links. The V2V reliability is evaluated by the average access success probability. The channel model uses large-scale fading and shadowing to statistically represent blockage effects, matching real-world obstructed propagation in urban intersection simulations under different building densities.

Figure 12 shows the total V2I throughput versus vehicle velocity under different speed ranges. The speed range gradually increases from [5, 10] m/s to [45, 50] m/s. The results show that for each density setting, the total V2I throughput varies only slightly as the velocity increases from [5, 10] m/s to [45, 50] m/s, indicating that the learned policy is not overly sensitive to mobility. Figure 13 reports the average V2V access success probability versus vehicle velocity. The success probability stays nearly constant across the entire speed range, demonstrating robust reliability under faster channel dynamics and denser spectrum reuse interference.

5.4. Statistical Analysis

To assess the statistical significance of performance differences among the five algorithms (RANDOM, DDPG, MADQN, MADDPG, and MADDPG-A2C) across the three key performance indicators—reward, V2I throughput, and V2V success probability—a statistical test analysis was conducted. The experimental results are shown in

Table 4. ANOVA statistical significance test results of five algorithms on key performance metrics (α = 0.05).

Methods	Reward	V2I Links’ Throughput (Mbps)	V2V Links’ Success Probability	MFLOPS (M)
RANDOM	85.09 ± 2.65	29.24 ± 0.87	0.797 ± 0.023	—
DDPG	64.47 ± 1.63	27.43 ± 0.42	0.688 ± 0.015	0.103
MADQN	69.65 ± 3.40	27.86 ± 0.62	0.735 ± 0.031	0.207
MADDPG	87.59 ± 1.54	29.80 ± 0.46	0.837 ± 0.021	0.43
MADDPG-A2C	88.75 ± 1.52	30.84 ± 0.40	0.851 ± 0.014	0.516
ANOVA p	1.15 × 10−216	1.19 × 10−288	8.62 × 10−198
ANOVA F	1297.14	285.27	1294.17

Black bold indicates the best performance metric.

.

The overall difference test was conducted using one-way analysis of variance (ANOVA, α = 0.05). The results showed that the p-values of reward (p = 1.15 × 10⁻¹⁶, F = 1297.14), V2I link throughput (p = 1.19 × 10⁻²⁸⁸, F = 285.27), and V2V success probability p = 8.62 × 10⁻¹⁹⁸, F = 1294.17) were all much smaller than 0.05. The F-values indicated significant differences between groups, confirming that there were statistically significant overall differences in the three indicators among different algorithms. This indicates that the differences among different algorithms in each indicator are significant, and different methods have statistical distinguishability in performance.

After confirming the overall significant differences among algorithms under each indicator through the ANOVA test, we further conducted post hoc pairwise comparison tests. The comprehensive results showed that MADDPG-A2C performed best in reward, V2I throughput, and V2V success probability, indicating its advantage in improving overall performance.

We further tested the actual computational overhead of different algorithms through experiments, with the results shown in Table 4. The overhead of the original MADDPG is 0.43 MFLOPS, and that of the MADDPG-A2C proposed in this paper is 0.516 MFLOPS. This additional overhead is mainly caused by the forward propagation of the state value network, which is consistent with the results of the theoretical complexity analysis. However, the 0.516 MFLOPS overhead is much lower than the computing power of mainstream in-vehicle edge computing chips and can fully meet the 100 ms level decision delay requirements specified for vehicular network environments in the simulation parameters.

6. Conclusions

This paper studies the dynamic resource allocation problem in vehicular networks, formulates an MADRL objective optimization problem under multi-demand constraints, and proposes the MADDPG-A2C strategy to address the convergence issues faced by traditional reinforcement learning algorithms in high-speed dynamic environments. By designing a multidimensional reward function, it effectively adapts to environmental changes and enhances learning efficiency. The proposed algorithm exhibits excellent scalability and generalization ability, which offers a positive outlook for its research prospects. In the future, a hybrid architecture combining prioritized experience replay and hierarchical experience replay mechanisms will be explored and developed. Meanwhile, the risk of overfitting in high-dynamic scenarios will be considered to balance the training distribution and enhance the model’s generalization ability.