AI-Driven Traffic Control Method and Reliability Analysis for Digital City Local Narrow-Road, Dense-Network

Abstract

In urban environments characterized by narrow roads and dense networks with short intersection spacing and high connectivity, traffic flows exhibit strong spatiotemporal coupling and pose safety challenges. Conventional traffic signal control approaches are difficult to achieve effective regional coordination, while existing control models based on artificial intelligence (AI) lack consideration for trustworthiness and robustness. To address these challenges, an AI-driven traffic control method for digital city traffic signals is proposed. A unified and decodable latent action representation space is constructed, in which the dependency between phase selection and green time duration is captured using discrete action embedding tables and a conditional variational autoencoder (CVAE), ensuring the stability and interpretability of the AI-driven model. Building on this foundation, a globally shared latent representation is integrated with a local coordination mechanism, and the proximal policy optimization (PPO) algorithm is employed for policy training. A state residual prediction regularization loss is introduced to improve the model’s generalization capability and convergence efficiency. Experiments were conducted using a real-road network and traffic flow data from the Rongdong District of Xiongan New Area. Under spatially imbalanced peak hour traffic conditions, the model reduced average vehicle delay by 14.84% and average queue length by 9.2%; under temporally imbalanced peak hour traffic, it achieved reductions of 5.36% and 7.2% in delay and queue length, respectively. These results demonstrate that the proposed method significantly enhances both traffic efficiency and system robustness, offering scalable, reliable technical support for secure and intelligent transportation systems (ITSs).

1. Introduction

With the accelerating pace of global urbanization, the number of motor vehicles has grown exponentially worldwide [1,2]. To address the increasing pressure on limited urban land resources, the narrow-road and dense-network structure has gradually been established as a dominant urban road network configuration. This configuration is typically characterized by short intersection spacing, high intersection density, and strong road connectivity. This leads to a severe imbalance between rapidly expanding traffic demand and limited road infrastructure capacity, resulting in poor travel experiences and reduced urban efficiency. Traditional traffic signal control relies on predefined timing schemes or basic adaptive logic, which have fixed control parameters and inflexible decision-making. They do not support high spatiotemporal imbalances. During peak hours, excessive vehicle queuing and reduced efficiency occur, while off-peak periods waste green light time on low traffic volumes. Particularly in dense intersection groups within narrow-road, dense-network environments, traditional techniques overlook the strong correlation between intersections. Consequently, it is difficult to meet modern cities’ sophisticated and intelligent traffic management needs. This creates a demand for AI-driven traffic signal control methods that can effectively ensure trustworthiness and robustness and meet the reliability needs of secure ITS.

In recent years, emerging technologies like connected and autonomous vehicles, 5G communication, artificial intelligence, and big data analytics have advanced urban transportation systems for traffic management. The trustworthiness and security of AI-driven signal control models have become critical for secure ITS operations. As a key ITS component, intelligent upgrading of traffic signal controls is essential for improved road network operation efficiency and security. Compared with traditional methods, intelligent traffic signals can perceive and adapt to real-time traffic using multi-source data and intelligent algorithms, overcoming traditional shortcomings [3]. However, existing reinforcement learning (RL) methods still have limited action space modeling and adaptability, especially for strongly coupled intersections in narrow-road, dense-network environments.

Specifically, current model-based traffic signal control methods focus on mathematical programming, in which the model is equipped to formulate multi-objective optimization problems, and algorithms seek optimal solutions within given constraints. However, these methods struggle to provide timely, accurate, and secure optimal control under complex traffic conditions. To address these limitations, deep reinforcement learning (DRL) [4] has been introduced to approximate value functions and policy gradients, enabling the system to capture high-order nonlinear dynamics of traffic flow through continuous interactions with the environment and to adaptively explore more efficient and robust control strategies.

Previous studies [5,6] have validated the effectiveness of DRL in optimizing single-intersection traffic control. However, directly extending DRL methods to continuous intersections presents notable adaptability challenges. The main difficulty lies in the strong coupling of traffic flows between adjacent intersections, which becomes more pronounced as the distance between them decreases. Vehicles entering from upstream intersections influence downstream operations, while downstream signal decisions in turn restrict upstream vehicle discharge, potentially leading to network-wide congestion. Moreover, the design of interaction mechanisms among multiple agents remains unsystematic. Existing single-agent approaches often fail to achieve global coordination, resulting in conflicting control actions and uneven resource allocation. Consequently, multi-agent reinforcement learning (MARL) has emerged as a promising research direction for urban continuous intersection control, aiming to overcome these limitations and enhance overall network efficiency. Notably, the proposed model is designed as a trustworthy AI system, with strong stability, robustness, and decision interpretability. Its performance is systematically evaluated through reliability analysis, meeting the security requirements of intelligent traffic signal control systems.

Based on the above analysis, four critical research gaps are identified: (1) existing RL methods fail to capture the dependency between discrete phase selection and continuous green time; (2) discrete and continuous actions are treated independently in hybrid action space approaches, leading to suboptimal performance; (3) most existing studies lack consideration for the trustworthiness and robustness of the AI-driven control model; and (4) multi-agent coordination under strong spatiotemporal coupling remains underdeveloped. To bridge these gaps, this study aims to develop a trustworthy AI-driven traffic signal control method. Accordingly, the core research question is as follows: How can a reliable AI-driven traffic signal control method be designed for narrow-road, dense-network environments to overcome the limitations of existing RL approaches in terms of action dependency, multi-agent coordination, and trustworthiness?

To address the aforementioned core challenges of signal control at continuous intersections, this study proposes an AI-driven multi-agent traffic signal control framework and conducts the reliability analysis focusing on action representation and collective intelligence control for multi-agent systems in narrow-road, dense-network environments. The core innovations and contributions are as follows:

• A discrete action embedding table and a CVAE [7] are employed to construct a latent representation space for discrete and continuous actions, while the PPO algorithm is applied for latent policy learning. Additionally, a nearest-neighbor search algorithm and a CVAE decoder conditioned on states and discrete action embeddings are introduced to reconstruct the discrete and continuous actions corresponding to the latent policy, enabling bidirectional conversion between original actions and their latent representations.
• Targeting the collective intelligence control requirements for continuous intersections, a globally shared latent action representation space and a locally shared state mechanism are designed. The state space incorporates the queue lengths of both upstream and downstream intersections. Guided by adjacent collaboration rewards, each agent achieves global cooperation while making independent decisions. This approach facilitates latent information interaction between agents and ensures consistent decision logic, thereby improving cooperative efficiency.
• To enhance the CVAE’s capability in learning the representation space of latent actions that dynamically capture the specificity of narrow-road, dense-network environments and improve the model’s generalization, a regularization loss function based on state residual prediction is incorporated. This assigns environmental impact-aware semantic information to each latent action, enabling agents to achieve a more in-depth comprehension of the higher-dimensional state space, thereby guiding the parameter updates of the hybrid action representation (HyAR).
• To improve model convergence efficiency and minimize training redundancy, a three-stage training process is implemented: HyAR pretraining, RL policy pre-training, and policy training. During the RL policy pre-training stage, a supervised learning loss function is incorporated to guide the preliminary updating of actor network parameters, thereby mitigating the risk of failing to achieve global optimization due to the high computational complexity inherent in high-dimensional HyAR spaces.

The remainder of this study is organized as follows. Section 2 reviews the related work on traffic signal control. Section 3 first presents the problem description for continuous signalized intersections and the DRL environment construction process, then introduces the HyAR module, RL policy module, and training methodology. Section 4 conducts experiments and analyzes the results. Section 5 concludes the study.

2. Related Work

RL-based traffic signal control methods are mainly categorized into value-based and policy-based approaches. Value-based methods, represented by Deep Q-Network (DQN) and its variants [8,9], approximate the action-value function to guide optimal action selection. Zhang et al. [10] proposed a signal control method based on Double Deep Q-Network (DDQN) for a single intersection. An attention mechanism is added to enhance important state features, and a weight update strategy is used to reduce action-value underestimation. These methods can make efficient decisions in discrete action spaces but may lead to loss of information and suboptimal decisions in high-dimensional or continuous action spaces. In contrast, policy-based methods, such as PPO [11] and Advantage Actor–Critic (A2C) [12], directly parameterize the policy via gradient ascent. They show superior stability and adaptability within complex action spaces, making them widely used in traffic signal timing optimization. Nevertheless, policy-based methods typically rely on extensive interaction in large-scale networks, and few studies consider the robustness and trustworthiness of these AI-driven models, which is a key gap in secure intelligent transportation research.

From a different perspective, RL algorithms can be classified as on-policy or off-policy, depending on whether the behavior policy for data sampling aligns with the target policy used for evaluation. Off-policy methods such as Q learning, DQN, and Deep Deterministic Policy Gradient use experience replay to reuse past data and increase data efficiency [13]. On-policy methods like PPO and State–Action–Reward–State–Action (SARSA) update policies with data generated by the current policy so that the training data fits the policy [14]. Yen et al. [15] employed an on-policy SARSA framework with Traffic Flow Maps to describe states in a dueling network. The study kept the on-policy setting and also added experience replay, which is often used in off-policy methods to improve data efficiency. Off-policy methods still have problems because the stored data is not the same as the latest policy, which can cause unstable training and slow learning, especially when traffic changes frequently.

The action space design is another key factor for RL-based traffic signal control. In current systems, the actions of DRL agents are of two types: choosing signal phases or setting green light time. Zheng et al. [16] and Zhou et al. [17] used fixed signal phases with set time intervals, which struggle to adapt well to dynamic traffic. Excessively short intervals may induce frequent phase transitions, increasing vehicle acceleration-deceleration losses and hindering pedestrian crossing demands. Conversely, extended intervals can cause inefficient green-light utilization and diminished overall traffic throughput. Zeng et al. [18] and Liang et al. [19] defined actions as determining the green light duration for the current phase within a predefined fixed phase sequence. When traffic volumes for different turning movements within the same signal phase show significant disparities, this can cause green light waste in low-flow directions. Signal control actions are insufficient to fully accommodate the spatiotemporal variations in traffic flow, limiting the agent’s ability to explore superior actions [20].

To overcome the limitations of single-action spaces, some researchers have proposed hybrid discrete-continuous action RL for signal control. Luo et al. [21] introduced a hybrid proximal policy optimization (HPPO)-based method for optimizing signal phase and timing at intersections, in which the agent learns discrete and continuous policies independently in two actor networks. However, this approach overlooks the intrinsic coupling between discrete and continuous actions at intersections. In real traffic control, the time required for each signal phase is different, which makes it hard for the agent to learn the best mixed actions. Bouktif et al. [22] proposed a hybrid action space using the Parameterized Deep Q-Network (P-DQN). The actor network creates green light times for each phase, and these values go into the critic network to choose the next phase. This method reduces the link problem between different actions, but using one network’s output as another’s input can make errors grow. When the hybrid action space becomes large, the system may also encounter scalability issues.

In application, traffic signal control algorithms can be divided into single-intersection and multi-intersection types. Research has gradually moved from optimizing individual intersections to controlling multiple intersections with technological advancement. Early studies worked on single-intersection control, where RL agents adjusted signal phases and timing to reduce local congestion [23,24]. With the rise of MARL, studies started to focus on multi-intersection control. Centralized MARL methods [25] use a global controller to optimize all intersections, ensuring global optimality but suffering from high computational complexity and poor scalability. Distributed MARL methods [26] let each intersection agent make its own decisions using local data. This improves scalability but often causes weaker global performance due to insufficient coordination among agents. Existing distributed methods lack effective mechanisms for information interaction and action consistency between agents, making congestion propagation in continuous intersection networks difficult to address. Table 1 reveals that existing methods lack effective handling of action dependency, multi-agent coordination, and system trustworthiness: gaps that the proposed HyAR-PPO method aims to address. Specifically, three major negative features of existing approaches are eliminated by the proposed method: (1) the inability to capture dependency between discrete phase selection and continuous green time; (2) weak coordination in multi-agent systems; and (3) the lack of trustworthiness and robustness in AI-driven control models.

Table 1. Comparison of RL-based traffic signal control methods.

Algorithm	Action Space	Coordination Mechanism	Evaluation
DDQN-Attention [10]	Discrete	Single-agent	Fixed discrete phases only; no continuous optimization
PPO-Discrete [11]	Discrete	Single-agent	Fixed interval; poor adaptation to dynamic flow
PPO-Continuous [11]	Continuous	Single-agent	Fixed phase sequence; uneven demand handling
SARSA [14,15]	Discrete	Single-agent/Multi-agent	On-policy; sample inefficient
HPPO [21]	Hybrid (independent)	Single-agent	Independent actions; ignores dependency
P-DQN [22]	Hybrid	Single-agent	Error propagation; limited scalability
Dueling DQN [9]	Discrete	Multi-agent	Weak coupling coordination
A2C [12]	Discrete	Multi-agent (centralized)	Centralized; high complexity; poor scalability
Centralized MARL [25]	Discrete	Centralized	Poor scalability
Distributed MARL [26]	Discrete	Decentralized	Weak coordination
HyAR-PPO	Hybrid (latent representation)	Decentralized with global latent sharing	Solves action dependency, and multi-agent coordination

Overall, RL-based traffic signal control has made substantial progress in algorithm types, data utilization modes, action space design, and application scenarios. However, significant gaps remain—few studies integrate the design of trustworthy AI-driven models into traffic signal control, and the robustness of AI-driven models in dynamic and disturbed traffic environments has not been fully explored. These gaps, combined with the urgent demand for secure ITS, provide the core entry point for this study.

3. HyAR-PPO-Based Traffic Signal Control Framework

3.1. Problem Description and Environment Modeling

This study focuses on the signal control problem in narrow-road, dense-network environments. Each signalized intersection is modeled as an independent agent. For any agent i, it can observe its own traffic state while also sharing partial local state information with adjacent intersection agent j to facilitate rapid response to merging upstream and downstream traffic flows and support collective decision-making. All agents are trained together in a network that features global parameter sharing. They choose signal phases and green light times from a combined action space. This design keeps decisions consistent and actions coordinated across all intersections. The research setup is shown in Figure 1.

Figure 1. The figure illustrates a scenario of continuous signalized intersections in urban traffic. The central agent i shares local states with adjacent agents j. Its action space consists of signal phase k and green light duration ${\tau }_k$, meaning the agent controls the traffic light by selecting $(k,{\tau }_k)$.

3.1.1. Hybrid Action Space

The hybrid action space can be formulated based on the Parameterized Action Markov Decision Process as proposed by Masson et al. [27], defined in Equation (1):

$$A=\{(k,x_k)|x_k\in X_k,\forall k\in K\}$$

(1)

where K represents the set of executable discrete actions for the agents. X_k denotes the set of continuous action parameters x_k associated with each discrete action k.

In this study, to provide the agents with more flexible options when deciding signal phases, the discrete action space K is defined as a set of eight non-conflicting phases, as illustrated in Figure 2. This discrete action space defines a probability distribution over the eight phases; at each decision step, the agent samples a phase according to this distribution rather than deterministically selecting one. Such a stochastic policy enables exploration during training, allowing the agent to try different phases and learn their expected rewards under varying traffic conditions. The continuous action x_k corresponds to the green light duration for the selected phase k, with its upper and lower bounds set to the maximum and minimum green times, respectively. Additionally, to ensure intersection clearance, a 3 s yellow light interval and a 1 s all-red interval are implemented before executing a new action whenever the next discrete phase differs from the current signal phase.

Figure 2. The figure shows eight non-conflicting signal phases, numbered from 1 to 8. Each phase represents a distinct set of traffic movements with no conflicting directions. The discrete action space K is defined as the eight phases, providing flexible options for the agent when selecting signal phases.

To address the limitations of existing RL methods in resolving scalability and action dependency issues in parameterized hybrid action spaces, HyAR reinforcement learning is proposed, as illustrated in Figure 3. This architecture builds one shared and decodable space for the agent’s understanding of the latent space. All intersection agents use this space to keep their action models consistent in the multi-agent system. Based on the current environmental state s_i,t, each agent i learns latent actions e_i and z_i,x with embedding dimensions d₁ and d₂ in the latent action representation space. These latent actions are then decoded to the original hybrid actions (k_i, x_i,k) to interact with the environment and get a reward r_i,t. HyAR includes the relation between continuous and discrete actions in the shared space, which is particularly critical for multi-agent coordination, because the shared latent space enables latent information interaction among agents. Specifically, each latent action corresponds to a semantically meaningful traffic control operation. The structured and shared latent space enables agents to consistently select similar latent actions under similar traffic conditions and to learn coordinated behavior without explicit communication. In this way, the hybrid action learning task becomes a continuous policy learning task in the latent space. The dimensionality of HyAR’s representation space (${ℝ}^{d_1+d_2}$) is lower than that of the parameterized hybrid action space (${ℝ}^{K+{\displaystyle \sum _{k\in K}\left|X_k\right|}}$), thereby avoiding introduced redundancy during policy learning and enhancing the model’s scalability.

Figure 3. The HyAR-based traffic signal control architecture is shown above. Multiple intersection agents interact with the environment. Each agent learns latent actions e_i and z_i,x with dimensions d₁ and d₂ in the latent space, then decodes them into hybrid actions including discrete phase k_i and continuous green time x_i,k to control the signal. Adjacent states and collaboration rewards are shared among agents for coordination.

3.1.2. Consistent Design of State Space and Reward Function

The state space is the set of environmental features perceived by the agent during its interaction with the environment in DRL, serving as the basis for formulating action policies and evaluating rewards. The reward function provides feedback after each action and helps guide the agent toward better states over time. Zheng et al. [28] demonstrated that simplified designs of the state space and reward function can enhance agent training performance. Building on this work, Bouktif et al. [29] further proposed a consistent design framework between the state space and reward function, enabling the agent to more accurately estimate the impact of each action on the state, thereby facilitating more rational decision-making.

In the RL environment of multiple continuous signalized intersections, in addition to considering each agent’s own state, it is also necessary to account for the influence of adjacent agents. Meanwhile, in the traffic signal control environment of continuous signalized intersections, agents do not engage in competitive behavior but instead collaborate with one another. Therefore, incorporating part of the traffic state information from adjacent agents as collective-shared state information enables agents to observe neighborhood states and make more globally informed decisions, thereby allowing the policy to approach the global optimum.

Regarding the selection of an agent’s local state, only two types of information are required for a complete description: the queue lengths at each approach lane and the current signal phase states. For the state observation of other agents, the impact of queue lengths in the approach lanes of downstream intersections must be considered. When adjacent signalized intersections experience congestion, the queues in their approach lanes connected to the exit lanes of the current intersection may spill back, leading to congestion propagation across intersections and potentially causing local traffic paralysis [30]. This inter-intersection spillback phenomenon is particularly pronounced under conditions of short intersection spacing. Thus, the neighborhood state information should necessarily include the queue lengths of the approach lanes of downstream intersections corresponding to the exit lanes of the current intersection. At this point, the state space can be expressed as Equation (2):

$$S_{i,t}=\left(p_{i,t,l},q_{i,t,l},O_{i,t,l^{\prime }}\right)$$

(2)

where S_i,t denotes the traffic state of intersection agent i at time t, including local state p_i,t,l, q_i,t,l and neighborhood state O_i,t,l′; p_i,t,l represents the signal phase information, taking a value of 0 or 1—1 indicates that approach lane l at intersection i is in a permitted phase at time t, while 0 indicates a prohibited phase; q_i,t,l is the queue length of approach lane l at intersection i at time t; o_i,t,l′ denotes the queue length of approach lane l′ at downstream intersections corresponding to each exit lane at current intersection i at time t.

Unlike single-agent RL, the design of the reward function in a multi-agent environment is more complex and critical. It not only serves as a measure of decision quality but also functions as the primary mechanism for coordinating multiple agents, providing feedback on group actions, and achieving global collaboration. To maintain consistency between each agent’s reward and its state space, the reward function for each agent is defined in Equation (3):

$$r_{i,t}=-r_{i,t+1}+q_{i,t}-{\displaystyle \sum _{j\in {\Lambda }_{i,t+1}}{q}_{j,t+1}}+\omega \cdot {\displaystyle \sum _{j\in {\Lambda }_{i,t+1}}{r}_{j,t+1}}$$

(3)

where r_i,t denotes the local reward of agent i at time t, consisting of three components: a phase-switching penalty, a queue length reward, and an adjacent collaboration reward; r_i,t,p is a phase-switching penalty following Wu et al. [31], taking a value of 1 if a phase switch occurs from the previous time step to the current time step, and 0 otherwise; q_i,t,l and q_i,t+1,L represent the queue lengths of approach lane l at intersection i at time t and t + 1, respectively; ${\tau }_{j,t,l}$ is the number of passing vehicles in approach lane l at the adjacent intersection j at time t; L is the set of approach lanes at the current intersection excluding dedicated right-turn lanes; J is the set consisting of agent j which are adjacent to the current agent in the environment; ω is a weight coefficient for adjacent collaboration rewards, set to 0.75 based on the empirical configuration demonstrated in [11] for balancing multi-objective reward functions in PPO-based control tasks.

3.2. HyAR Reinforcement Learning Model

As analyzed in Section 3.1.1, the HyAR space should account for the dependencies among the original hybrid actions and possess decodable characteristics. Specifically, the latent actions selected by the latent policy should encapsulate the dependencies within the hybrid actions and be mappable to the original hybrid action space for environmental interaction. Based on this rationale, an AI-driven policy training architecture with HyAR is proposed, comprising a HyAR and RL policy module, as illustrated in Figure 4.

Figure 4. This diagram shows a hybrid action RL framework for traffic signal control, featuring two core modules: a HyAR module that learns and decodes latent hybrid actions, and an RL policy module that trains in this latent space to generate actions for environment interaction.

Latent discrete and continuous actions are first extracted from the latent representation space by the PPO agent networks, which are then fed into the CVAE decoder. Based on the current traffic state and the discrete action embedding tables, the original hybrid actions (signal phase and green time) are reconstructed by the decoder. The reconstructed actions are executed by the agent in the simulation environment, after which rewards are received, and the transitions are stored in a replay buffer. The agent network is updated based on stored latent actions using the PPO loss function, while the representation module is updated periodically through CVAE and state residual loss. This cyclical process enables end-to-end training of the entire framework. Algorithm 1 summarizes the three-stage training procedure of the AI-driven traffic signal control framework. Stage 1 (lines 1–7) pre-trains the HyAR module by collecting random experiences to learn the latent representation space. Stage 2 (lines 8–11) pre-trains the PPO agent using supervised learning to map latent actions from the HyAR module. Stage 3 (lines 12–24) jointly trains the entire framework, where the PPO agent samples latent actions from the latent space, the HyAR module decodes them into original hybrid actions for environment interaction, and both modules are updated periodically using their respective loss functions. Finally, the trained policy is assessed using traffic efficiency metrics (line 25).

Algorithm 1 AI-Driven Traffic Signal Control

Input initial traffic signal agents; environment state space; hyperparameters $\epsilon ,\beta ,\mu$

1: Initialize traffic signal agents

2: for each agent i do

3:      Generate random state $S_{i,t}$

4:      Select signal phase $k_i$ and green time $x_{i,t}$

5:      Execute action and obtain state residual ${\delta }_i$

6:      Train HyAR module to learn latent representation $e_i$

7: end for

8: for each agent i do

9:      Obtain latent actions $e_i$ and ${\widehat{z}}_{i,x}$ from HyAR

10:    Train PPO agent to learn actions in latent space

11: end for

12: while training not complete do

13:    for each agent i do

14:        Observe state $S_{i,t}$

15:        Select latent actions $e_i$ and ${\widehat{z}}_{i,x}$ from HyAR

16:        Decode to original actions ($k_i$,$x_{i,t}$)

17:        Execute action and collect reward ri,t

18:        Store experience in replay buffer

19:        Calculate HyAR loss LHyAR and update module

20:        Calculate PPO loss LPPO and update policy

21:        Update state residual ${\delta }_i$ = $S_{i,t}$ − $S_{i,t+1}$

22:        Compute state residual loss to guide learning

23:    end for

24: end while

25: Assess traffic efficiency and control stability

Output optimized traffic signal control policy

3.2.1. HyAR Module

To achieve accurate representation and decoding of hybrid actions, a HyAR module based on a discrete action embedding table and a CVAE architecture is designed. On one hand, a discrete action embedding table $E_{\xi ,K}\in {ℝ}^{K\times d_1}$ with learnable parameters $\zeta$, shared among all agents, is established to facilitate the conversion of discrete actions between the original space and the representation space. Specifically, a discrete signal phase $k_i$ selected by intersection agent i can obtain its latent discrete action encoding $e_{\xi ,i,k}$ by indexing the k-th row of $E_{\xi ,K}$. Conversely, a latent discrete action $e_i$ learned by the latent policy can be mapped back to a discrete signal phase ${\widehat{k}}_i$ by applying a nearest-neighbor search technique to find the most similar latent discrete action encoding $e_{\xi ,i,k}$ within $E_{\xi ,K}$ and retrieving its corresponding phase ${\widehat{k}}_i$, ensuring consistent decoding of discrete actions in the multi-agent environment. On the other hand, a CVAE is employed to construct a latent representation space for continuous action parameters. During the HyAR training phase, the environmental state $s_{i,t}$ and the latent discrete action encoding $e_{\xi ,i,k}$ of the agent i, serving as conditional features, along with the phase duration $x_{i,k}$ as the target feature, are input into the encoder together, which outputs the parameters of the variational posterior distribution—namely, the mean vector ${\mu }_{i,x}$ and the standard deviation vector ${\sigma }_{i,x}$. The latent continuous action ${\widehat{z}}_{i,x}$ is sampled according to Equation (4).

$${\widehat{z}}_{i,x}={\mu }_{i,x}+{\sigma }_{i,x}\times \epsilon$$

(4)

where $\epsilon$ is a random variable following a standard Gaussian distribution, i.e., $\epsilon \sim N\left(0,1\right)$.

Following sampling by the encoder, the latent continuous action ${\widehat{z}}_{i,x}$, along with the conditional feature encodings, is fed into the CVAE decoder to obtain the reconstructed phase green light duration ${\widehat{x}}_{i,k}$. The discrete action embedding table parameters $\zeta$, as well as the CVAE encoder parameters $\psi$ and decoder parameters $\phi$, are updated by minimizing a loss function. This can be expressed as Equations (5)–(7):

$$L^{\mathrm{CVAE}}\left(\zeta ,\psi ,\phi \right)=E_{i,t}\left\{L_i^{\mathrm{recon}}\left[p_{\phi }\left({\widehat{x}}_{i,k}\left|{\widehat{z}}_{i,x},s_{i,t},e_{\zeta ,i,k}\right.\right)\right]-D_{i,\mathrm{KL}}\left[q_{\psi }\left({\widehat{z}}_{i,x}\left|x_{i,k},s_{i,t},e_{\zeta ,i,k}\right.\right)‖N\left(0,1\right)\right]\right\}$$

(5)

$$L_i^{\mathrm{recon}}\left[p_{\phi }\left({\widehat{x}}_{i,k}\left|{\widehat{z}}_{i,x},s_{i,t},e_{\zeta ,i,k}\right.\right)\right]={\left(x_{i,k}-{\widehat{x}}_{i,k}\right)}^2$$

(6)

$$D_{i,\mathrm{KL}}\left[q_{\psi }\left({\widehat{z}}_{i,x}\left|x_{i,k},s_{i,t},e_{\zeta ,i,k}\right.\right)‖N\left(0,1\right)\right]=-0.5\times \left(1+\log {\sigma }_{i,x}^2-{\mu }_{i,x}^2-{\sigma }_{i,x}^2\right)$$

(7)

where $L^{\mathrm{CVAE}}\left(\zeta ,\psi ,\phi \right)$ is the CVAE loss function adapted from Sadeghi et al. [7]; $q_{\psi }\left({\widehat{z}}_{i,x}\left|x_{i,k},s_{i,t},e_{\zeta ,i,k}\right.\right)$ is the conditional variational posterior distribution approximated by the encoder; $p_{\phi }\left({\widehat{x}}_{i,k}\left|{\widehat{z}}_{i,x},s_{i,t},e_{\zeta ,i,k}\right.\right)$ is the parameterized conditional likelihood of the decoder; $E_{i,t}$ denotes the expectation over the batch of training samples for agent i at time t; $L_i^{\mathrm{recon}}\left[p_{\phi }\left({\widehat{x}}_{i,k}\left|{\widehat{z}}_{i,x},s_{i,t},e_{\zeta ,i,k}\right.\right)\right]$ is the reconstruction loss, representing the mean squared error between the original green light duration $x_{i,k}$, for phase k_i selected by agent i, and the reconstructed phase green light duration ${\widehat{x}}_{i,k}$ output by the encoder–decoder process; and $D_{i,\mathrm{KL}}\left[q_{\psi }\left({\widehat{z}}_{i,x}\left|x_{i,k},s_{i,t},e_{\zeta ,i,k}\right.\right)‖N\left(0,1\right)\right]$ is the Kullback–Leibler divergence between the conditional variational posterior distribution and the standard Gaussian prior distribution, used to quantify the similarity between the two distributions—a smaller D_i,KL value indicates that the variational posterior distribution is closer to the desired prior distribution, leading to more stable representations of the latent continuous action.

However, updating parameters solely via the CVAE loss may result in latent actions of agent i lacking knowledge of the specific dynamics of its local environment, potentially leading to model overfitting. Inspired by unsupervised environment dynamics prediction methods [32], this study introduces a regularization loss function based on state residual prediction. This loss assigns semantic information about the impact of each latent action of agent i on the local environment, thereby guiding the learning of parameters within the HyAR module. Furthermore, this state residual information enables agent i to better grasp the potential correlation between the queue length of the current intersection q_i,t,l, that of downstream intersections o_i,t,l′ and the signal phase p_i,t,l, thereby facilitating the inference of the impact of each latent action on the current local states and neighborhood states from the representation space of latent actions. As shown in the decoder part of Figure 4, a sub-network is appended after the encoded vector containing all conditional features to predict the state residual of agent i, which is formulated as Equations (8) and (9):

$${\delta }_{i,t}=s_{i,t+1}-s_{i,t}$$

(8)

$${\widehat{\delta }}_{i,t}=p_{\phi }\left({\widehat{\delta }}_{i,t}\left|{\widehat{z}}_{i,x},s_{i,t},e_{\zeta ,i,k}\right.\right)$$

(9)

where ${\delta }_{i,t}$ and ${\widehat{\delta }}_{i,t}$ are the actual and predicted state residual vectors of agent i at time t, respectively; $s_{i,t+1}$ represents the state space of agent i at the next time step t + 1.

The total HyAR loss function, incorporating the state residual prediction, is computed in Equations (10) and (11):

$$L^{\mathrm{HyAR}}\left(\zeta ,\psi ,\phi \right)=L^{\mathrm{CVAE}}\left(\zeta ,\psi ,\phi \right)+\beta \cdot L^{\mathrm{State}}\left(\zeta ,\psi ,\phi \right)$$

(10)

$$L^{\mathrm{State}}\left(\zeta ,\psi ,\phi \right)=E_{i,t}\left[{‖{\delta }_{i,t}-{\widehat{\delta }}_{i,t}‖}_2^2\right]$$

(11)

where $L^{\mathrm{HyAR}}\left(\zeta ,\psi ,\phi \right)$ is the total HyAR loss function; $\beta$ is a weighting hyperparameter; and $L^{\mathrm{State}}\left(\zeta ,\psi ,\phi \right)$ is the expected state residual prediction regularization loss function for each agent, representing the mean squared error between ${\delta }_{i,t}$ and ${\widehat{\delta }}_{i,t}$. By training the HyAR module to predict ${\widehat{\delta }}_{i,t}$, the model learns to associate each latent action with its expected traffic impact. For example, a latent action that extends green time for a congested entrance lane should predict a larger queue reduction than a suboptimal action. This regularization connects the abstract latent space to physical traffic dynamics, improving generalization and robustness under spatiotemporally imbalanced conditions.

3.2.2. Reinforcement Learning Policy Module

HyAR transforms the original hybrid action policy learning problem into a continuous policy learning task within the latent action representation space. As illustrated in the RL policy module of Figure 4, this study employs the PPO method to learn latent actions of each agent in the representation space. As a policy-based DRL method, PPO restricts the update magnitude of the policy network through a clipping mechanism and enables multiple reuses of experience data. In the parallel training of multiple agents, it can not only balance the learning efficiency of each agent but also ensure the stability and reliability of policy updates, adapting to the requirements of distributed decision-making scenarios.

Inspired by the optimization approach using two parallel actor networks in HPPO, a globally shared latent discrete action actor network and a latent continuous action actor network are constructed. Following the standard PPO clipped surrogate objective from Schulman et al. [11], each agent independently learns the two policies, $\pi \left(e_i\left|s_{i,t}\right.\right)$ and $\pi \left(z_{i,x}\left|s_{i,t}\right.\right)$, in the representation space based on the shared parameters. The policy loss functions for the latent discrete and continuous action policies are presented in Equations (12) and (13):

$$L^{\mathrm{Clip}}\left({\theta }_{\mathrm{d}}\right)=-E_{i,t}\left\{\min \left\{u_{i,t}\left({\theta }_{\mathrm{d}}\right){\widehat{A}}_{i,t}\left(s_{i,t},e_i\right),\mathrm{Clip}\left[u_{i,t}\left({\theta }_{\mathrm{d}}\right),1-{\epsilon }_1,1+{\epsilon }_1\right]{\widehat{A}}_{i,t}\left(s_{i,t},e_i\right)\right\}\right\}$$

(12)

$$L^{\mathrm{Clip}}\left({\theta }_{\mathrm{c}}\right)=-E_{i,t}\left\{\min \left\{u_{i,t}\left({\theta }_{\mathrm{c}}\right){\widehat{A}}_{i,t}\left(s_{i,t},z_{i,x}\right),\mathrm{Clip}\left[u_{i,t}\left({\theta }_{\mathrm{c}}\right),1-{\epsilon }_2,1+{\epsilon }_2\right]{\widehat{A}}_{i,t}\left(s_{i,t},z_{i,x}\right)\right\}\right\}$$

(13)

where $L^{\mathrm{Clip}}\left({\theta }_{\mathrm{d}}\right)$ and $L^{\mathrm{Clip}}\left({\theta }_{\mathrm{c}}\right)$ are the policy gradient loss functions for the latent discrete and latent continuous actor networks, respectively; ${\theta }_{\mathrm{d}}$ and ${\theta }_{\mathrm{c}}$ are the parameters of the latent discrete and latent continuous actor networks, respectively; $u_{i,t}\left({\theta }_{\mathrm{d}}\right)$ and $u_{i,t}\left({\theta }_{\mathrm{c}}\right)$ are the probability ratios of selecting actions e_i and z_i,x for agent i in state s_i,t between the new and old policies, which are used to constrain the magnitude of the new policy update; Clip is the function that confines $u_{i,t}\left({\theta }_{\mathrm{d}}\right)$ and $u_{i,t}\left({\theta }_{\mathrm{c}}\right)$ within the intervals $\left(1-{\epsilon }_1,1+{\epsilon }_1\right)$ and $\left(1-{\epsilon }_2,1+{\epsilon }_2\right)$, respectively, where ${\epsilon }_1$ and ${\epsilon }_2$ are the Clip hyperparameters; ${\widehat{A}}_{i,t}$ is the advantage function for agent i to execute actions e_i and z_i,x in state s_i,t as proposed by Schulman et al. [11], generally computed using the Generalized Advantage Estimation [33], expressed as Equation (14):

$${\widehat{A}}_{i,t}={\displaystyle \sum _{\iota =0}^{T-1}{\left(\gamma \lambda \right)}^{\iota }\left[r_{i,t+\iota }+\gamma V\left(s_{i,t+\iota +1}\right)-V\left(s_{i,t+\iota }\right)\right]}$$

(14)

where T is the sampling timestep; $\iota$ is the timestep counting index after time t; $\gamma$ is the discount factor for future return; $\lambda$ is the decay rate for the contribution over steps; and $V\left(s_{i,t}\right)$ is the state-value function, representing the expected discounted return for agent i from state s_i,t at time t, as shown in Equation (15):

$$V\left(s_{i,t}\right)=E_t\left({\displaystyle \sum _{T>t}{\gamma }^{T-t}{r}_{i,t}}\right)$$

(15)

within the policy network architecture of the PPO method, a critic network is utilized to estimate the state-value function. The learned state-value function is subsequently used to derive the estimated Generalized Advantage Estimation, with its loss function for policy gradient updates defined in Equation (16):

$$L^{\mathrm{Value}}\left(\eta \right)=E_{i,t}{\left[V\left(s_{i,t}\right)-V_{\eta }\left(s_{i,t}\right)\right]}^2$$

(16)

where $L^{\mathrm{Value}}\left(\eta \right)$ is the loss function for policy gradient updates of the critic network; $\eta$ represents the shared parameters of the critic network; and $V_{\eta }\left(s_{i,t}\right)$ denotes the state-value function estimated by the critic network.

3.2.3. Reliable Training Process

Traditional RL-based traffic signal control methods often suffer from instability during the initial training phase, as they typically employ random policies to interact with the environment. Data collected under these conditions is unsuitable for training the HyAR module. Furthermore, RL policies encounter challenges of high computational complexity and risk converging to local optima when dealing with high-dimensional latent action representation spaces. Therefore, to improve convergence efficiency and reduce training redundancy, a three-stage training process is designed to improve AI-driven convergence reliability. The complete training process is illustrated in Figure 5.

Figure 5. The training process based on HyAR reinforcement learning is divided into three sequential stages. First, the HyAR module is pre-trained using experience data collected by a pre-defined algorithm. Next, the reinforcement learning policy network is pre-trained on the learned latent action space. Then the full model is jointly fine-tuned, with both the HyAR module and the policy network updated simultaneously using collected environmental experiences.

During the HyAR pre-training stage, a pre-trained HPPO algorithm is employed to collect the discrete signal phase k_i and its corresponding green light duration x_i,k taken by each intersection agent under random states s_i,t. Through interaction with the environment, the state residual vector ${\delta }_i$ resulting from executing the hybrid action $\left(k_i,x_{i,k}\right)$ is acquired. The HyAR module then learns the latent dependencies among the hybrid actions and subsequently updates the parameters of $E_{\zeta ,K}$ and the CVAE.

In the RL policy pre-training stage, a dual continuous actor network PPO method is applied to learn the latent actions from the HyAR pre-training policy. The network parameters are pre-updated by minimizing the following loss function, as shown in Equation (17):

$$L^{\mathrm{Pre}}\left({\theta }_d,{\theta }_c\right)=E_{i,t}\left[{‖e_i-e_{\zeta ,i,k}‖}_2^2\right]+E_{i,t}\left[{‖z_{i,x}-{\widehat{z}}_{i,x}‖}_2^2\right]$$

(17)

where $L^{\mathrm{Pre}}\left({\theta }_d,{\theta }_c\right)$ is the loss function for policy gradient updates of the actor networks during the RL pre-training stage, composed of the mean squared error from the latent discrete actor network and the mean squared error from the latent continuous actor network; $\left(e_{\zeta ,i,k},{\widehat{z}}_{i,x}\right)$ and $\left(e_i,z_{i,x}\right)$ are the latent hybrid actions output by the HyAR pre-training policy and the RL policy, respectively, under the random state space s_i,t of agent i.

During the policy training stage, the latent policy outputs the latent action $\left(e_i,z_{i,x}\right)$ for each agent under the random state space s_i,t, which is then decoded by the HyAR module into the original hybrid action $\left(k_i,x_{i,k}\right)$ for interacting with the environment to collect the reward r_i,t and the state residual vector ${\delta }_i$, which are stored in the experience replay buffer. The AI-driven HyAR-PPO method updates the network parameters of both the HyAR module and the RL policy module by performing importance sampling on data from the experience replay buffer.

4. Experimental Results and Discussion

4.1. Experimental Scenario Selection and Initialization

To validate the feasibility and effectiveness of the AI-driven HyAR-PPO method in narrow-road, dense-network environments, intersections within the Rongdong area Digital Roads in Xiongan New Area were selected for testing. The evaluation focused on three consecutive intersections along the main road, Lemin Street, which share typical features including short spacing, high traffic volume, and directional imbalance. These intersections were: (1) Shanrong Road and Lemin Street, (2) Fujia Road and Lemin Street, and (3) North Chongwen Road and Lemin Street. The distances between adjacent intersections were approximately 190 m and 200 m, respectively.

The spatial imbalance effect of traffic flow across different approach lanes was simulated using real traffic data proportions from the evening peak hour. Additionally, to emulate the temporal imbalance effect of traffic flow, Poisson-distributed traffic flow data corresponding to off-peak and peak hours were introduced during different time intervals. Finally, the testing environment was constructed using the TraCI interface of the SUMO (Simulation of Urban Mobility) simulation platform, enabling real-time acquisition of traffic flow information and testing of traffic signal control. The actual experimental scenario is illustrated in Figure 6.

Figure 6. Test scenario of Rongdong area Digital Roads in Xiongan New Area. (a) Online map from Bing Maps (Microsoft Corporation, Redmond, WA, USA) [34] and (b) schematic diagram of the simulation scenario on SUMO.

According to the actual traffic data provided by China Telecom Digital City Technology Co., Ltd., the original traffic data of the three intersections vary. Given the inherent randomness of Poisson-distributed vehicle generation (over 15 min intervals), the values were averaged and rounded to the nearest hundred, yielding 900 veh·h⁻¹ for off-peak hours and 1500 veh·h⁻¹ for peak hours on the east–west main road of Lemin Street. Therefore, simulation test scenarios for spatially unbalanced traffic flow during off-peak and peak hours were constructed according to the turning proportions summarized in Table 2, alongside temporally imbalanced traffic simulation test scenarios under spatially balanced distributions, as presented in Table 2 and Table 3.

Table 2. Traffic flow proportion of each approach lane.

Name of Intersection	Traffic Flow Classification	North Approach Lane		East Approach Lane		South Approach Lane		West Approach Lane
		Straight	Left	Straight	Left	Straight	Left	Straight	Left
Intersection of Shanrong Rd. and Lemin St.	Authentic traffic flow proportion	0.3036	0.3429	0.8186	0.1002	0.3388	0.3224	0.7933	0.0772
	Traffic flow proportion under spatial balance	0.5	0.25	0.5	0.25	0.5	0.25	0.5	0.25
Intersection of Fujia Rd. and Lemin St.	Authentic traffic flow proportion	0.5413	0.2783	0.8914	0.0407	0.5405	0.1081	0.7844	0.1367
	Traffic flow proportion under spatial balance	0.75	0.125	0.75	0.125	0.75	0.125	0.75	0.125
Intersection of North Chongwen Rd. and Lemin St.	Authentic traffic flow proportion	0.2674	0.5116	0.9266	0.0169	0.5962	0.1871	0.7643	0.1067
	Traffic flow proportion under spatial balance	0.625	0.1875	0.625	0.1875	0.625	0.1875	0.625	0.1875

Table 3. Traffic statistics by time period (veh·h⁻¹).

Traffic Flow Classification	Road Classification	Time Period
		[0, 15) min	[15, 30) min	[30, 45) min	[45, 60] min
Off-peak hours	East–west arterial road	720	900	1080	900
	North–south minor roads	240	300	360	300
Peak hours	East–west arterial road	1200	1500	1800	1500
	North–south minor roads	400	500	600	500

The neural network parameters of the AI-driven HyAR-PPO method were optimized in the aforementioned testing environment. In accordance with practical control requirements, the maximum and minimum green times for each phase were set to 45 s and 15 s, respectively. In addition, the loss weight, learning rate decay, and contribution decay coefficients were set to their default values in this study. Other model parameter configurations are presented in Table 4. Furthermore, comparisons were made with benchmark RL-based methods to validate the feasibility of the AI-driven HyAR-PPO method in handling complex and dynamic traffic scenarios.

Table 4. Hyperparameter settings.

Hybrid Action Representation Module		Reinforcement Learning Policy Module
Model Parameter	Set Value	Model Parameter	Set Value
Latent Discrete Action Dimension d1	6	Actor Network Learning Rate	0.0003
Latent Continuous Action Dimension d2	6	Critic Network Learning Rate	0.001
Learning Rate	0.0001	Discount Factor γ	0.99
Conditional Encoding Hidden Layer	256	Actor Network Hidden Layer	(256, 128, 128)
Continuous Action Encoding Hidden Layer	256	Critic Network Hidden Layer	(256, 128, 64)
Latent Action Encoding Hidden Layer	(128, 128)	Batch Size	256
Continuous Action Decoding Hidden Layer	(256, 258)	Number of Training Episodes per Batch	20
State Residual Decoding Hidden Layer	128	Clipping Hyperparameter	0.2

1.

Fixed-time control

This method adopts the traditional signal control method, the Webster method. The signal cycle is set to 80 s based on traffic demand. A two-phase control scheme is adopted: Phase 1 serves east–west through movements with 49 s of green, and Phase 2 serves north–south through movements with 25 s of green. The remaining 6 s are allocated to yellow and all-red intervals. No real-time adaptation is applied.

2.

PPO-Continuous [11]

This method employs PPO with a single continuous action space for controlling continuous green light duration under a fixed phase sequence. The study utilizes a symmetric phase sequence structure, with maximum and minimum green times of 60 and 15 s for each phase, respectively.

3.

PPO-Discrete [11]

This method employs PPO with a single discrete action space for signal phase selection control at fixed 15 s intervals. The study utilizes eight phases.

4.

HPPO [21]

This method uses a PPO-based model with a hybrid action space, where one discrete actor network chooses the next signal phase and one continuous actor network sets its green time. The system uses eight phases, and the green time is between 15 and 45 s.

4.2. Reliable Analysis for Experimental Results

Reliable analysis of AI-driven control models is as critical as the nominal performance in secure ITS, especially necessary in narrow-road, dense-network environments, where short intersection spacing amplifies the impact of disturbances. Therefore, repeated experiments under spatiotemporally imbalanced traffic conditions, including controlled disturbances like sudden traffic surges and state observation noise, are conducted to test the proposed AI-driven HyAR-PPO method’s reliability.

Before evaluating reliability, the nominal performance of the proposed HyAR-PPO method is first established. Through the derivation of the SUMO simulation data and performance analysis, changes before and after optimization are compared using multiple traffic evaluation metrics. Multi-metric improvements after optimization are illustrated in Figure 7, further validating the effectiveness and practicality of the AI-driven HyAR-PPO signal control strategy for coordinated optimization of continuous intersections.

Figure 7. Performance improvement of the proposed HyAR-PPO method across three coordinated intersections. The figure presents the relative improvements (%) in mean speed (km/h), waiting time (min), and waiting count after optimization.

The training convergence behavior of each method is analyzed under multiple spatiotemporally imbalanced scenarios. During model training, the DRL agent completed 440 episodes in a spatiotemporally imbalanced scenario with simulated evening peak-hour traffic, with each episode lasting 3600 s. The trends of average delay time and average queue length relative to training episodes across various methods are shown in Figure 8. The PPO method based on a hybrid action space achieves superior performance, exhibiting lower average vehicle delays and shorter average queue length compared to PPO methods with either single discrete or continuous action spaces. This result shows a clear benefit of hybrid action space DRL in complex and changing traffic. HyAR-PPO performs better than HPPO, reducing average delay time and queue length by 12.40% and 7.02%, indicating the efficiency of latent representations.

Figure 8. Comparison of evaluation metrics for different methods during the model training stage. (a) average delay time and (b) average queue length.

During model validation, the best training episode of each method is chosen, and each method is tested 20 times in simulation. The mean and standard deviation of these tests are shown in Figure 9 and Figure 10, displaying how the average delay time changes with time steps T under spatial and temporal imbalance. The PPO-Continuous results in Figure 9 demonstrate that the continuous green-time control with a fixed phase order cannot handle uneven traffic on different approaches. Comparison of the PPO-Discrete method under off-peak and peak flow conditions in Figure 10 reveals that the discrete phase selection control strategy with a fixed decision interval cannot track dynamic traffic fluctuations either. These results show that using only one type of action space limits DRL traffic signal control. When the two hybrid action methods are compared, HyAR-PPO has a lower mean and standard deviation of average delay in all four cases, which means it follows flow changes better, keeps control more stable, and is more robust.

Figure 9. Average delay time for each method under spatially unbalanced traffic scenarios. (a) Off-peak hour and (b) peak hour. Colored lines represent mean values, and colored shadows represent variance bands.

Figure 10. Average delay time for each method under temporally unbalanced traffic scenarios. (a) Off-peak hour and (b) peak hour. Colored lines represent mean values, and colored shadows represent variance bands.

The complete test results for all methods during the validation phase are summarized in Table 5. Compared to the hybrid action space RL method HPPO, HyAR-PPO reduces the average delay time by 7.73%, 14.84%, 2.57%, and 5.36% in the four scenarios: spatially imbalanced off-peak flow, spatially imbalanced peak flow, temporally imbalanced off-peak flow, and temporally imbalanced peak flow, respectively. Corresponding reductions in average queue length are 6.63%, 9.15%, 4.00%, and 7.17%, respectively. Notably, the optimization performance of AI-driven HyAR-PPO in temporally imbalanced traffic flow scenarios is slightly lower than in spatially imbalanced scenarios, and the improvement over HPPO is relatively smaller. This occurs because, in temporally imbalanced scenarios, the traffic flow distribution across all approaches is more balanced at any given moment, thereby reducing the impact of the dependency between discrete and continuous hybrid actions.

Table 5. Test results of each method in model validation stage.

Scenario	Evaluation Index	Fixed-Time	PPO- Continuous	PPO- Discrete	HPPO	Hyar-PPO	Optimization
Spatially imbalanced traffic scenarios during off-peak hours	Average delay time	95.03 s	80.51 s	38.68 s	35.98 s	33.20 s	7.73%
	Average queue length	7.5 veh	5.7 veh	2.5 veh	2.5 veh	2.2 veh	6.3%
Spatially imbalanced traffic scenarios during peak hours	Average delay time	155.32 s	127.41 s	80.01 s	61.37 s	52.26 s	14.84%
	Average queue length	12.5 veh	9.8 veh	5.9 veh	4.7 veh	4.3 veh	9.2%
Temporally imbalanced traffic scenarios during off-peak hours	Average delay time	45.71 s	38.62 s	37.41 s	35.04 s	34.14 s	2.57%
	Average queue length	3.6 veh	2.5 veh	2.4 veh	2.3 veh	2.2 veh	4.0%
Temporally imbalanced traffic scenarios during peak hours	Average delay time	85.74 s	63.28 s	70.80 s	59.35 s	56.17 s	5.36%
	Average queue length	7.9 veh	5.5 veh	6.2 veh	5.2 veh	4.8 veh	7.2%

The operational behavior of the proposed model under challenging traffic conditions is displayed by SUMO (v1.20.0) software to simulate traffic flow at three connected intersections. The trustworthy AI-driven HyAR-PPO model optimizes the signal control system and shows strong regional coordination, high stability, and good robustness under changing traffic. Figure 11 illustrates the dynamic coordination of signals by HyAR-PPO, achieving efficient green wave progression and improving road network efficiency. Efficient green wave progression is achieved, improving overall road network capacity and traffic flow efficiency.

Figure 11. A component of the simulation process based on SUMO. (a) The traffic flow reaches the uppermost intersection; (b) the traffic flow progresses to the second intersection; (c) the traffic flow reaches the third intersection; and (d) the traffic flow clears at the third intersection.

The proposed method is supported by real-world data collected from 19,625 intelligent road infrastructure devices deployed in the Rongdong area, including 163 traffic signal systems, 3840 cameras, 1109 radar sensors, 450 RSUs, seven edge computing nodes, 112 servers, and backhaul network equipment (22 OLTs and 1832 ONUs). With these diverse data sources, the method achieves comprehensive traffic state perception and reliable model training. It is computationally efficient for real-time signal control, with low inference latency that satisfies typical intersection operation requirements. It can be readily integrated with existing traffic sensor infrastructure and maintain stable performance under common measurement errors. These features support its practical deployment in digital city environments.

5. Conclusions

A trustworthy AI-driven collective traffic signal control method based on HyAR reinforcement learning is proposed for narrow-road, dense-network environments. The method targets the dynamic coupling between upstream and downstream intersections and the reliability requirements of secure ITS. By mapping original hybrid actions into a latent action representation space, the scalability and action dependency issues inherent in existing hybrid action traffic signal control methods are addressed. By combining three key components—the state residual prediction module, the reinforcement learning policy pre-training mechanism, and the optimization of latent hybrid action—training efficiency and overall model performance were both enhanced. Experiments on real traffic data from continuous signalized intersections in the Xiongan New Area’s Rongdong area showed the AI-driven HyAR-PPO method outperforming three benchmark models (PPO-Discrete, PPO-Continuous, and HPPO). It reduced average delay under spatially imbalanced peak flows by 58.98%, 34.68%, and 14.84%, respectively, and under temporally imbalanced peak flows by 11.24%, 20.66%, and 5.36%, respectively. These results demonstrate that the proposed method more effectively controls complex and dynamic traffic states compared to the benchmark methods, exhibiting clear practical value. It also provides a practical and reliable AI-driven framework along with a testing scheme for deploying secure intelligent traffic signal collective control systems in digital cities.

While the HyAR-PPO method demonstrates significant improvements, several limitations remain to be addressed. In future work, we plan to extend the method to larger-scale road networks to further evaluate its scalability. Graph neural networks are expected to be integrated for more effective spatiotemporal dependency modeling. The transferability of the strategies learned in dealing with unknown intersection layouts will also be further studied, which will reduce the need for retraining in new scenarios.