Abstract
In the field of urban traffic management, optimizing traffic signals at intersections is crucial for enhancing traffic flow efficiency. Despite advances in intelligent traffic signal control strategies through deep reinforcement learning (DRL), practical deployment challenges persist, such as abrupt changes in signal phases and significant hardware costs. This paper proposes a novel Traffic Characteristics-Guided Progressive optimization (TCGP) method that builds on classical fixed-time traffic signals. It is based on the classic fixed-time and quickly optimizes the green time ratio of intersection traffic lights by integrating the relationship between green light duration and traffic flow. Then, it efficiently explores the traffic signal cycle duration of a single intersection. Using a progressive optimization strategy, TCGP addresses the “curse of dimensionality” problem caused by a large number of intersections. TCGP ensures compatibility with traditional control methods and offers performance comparable to state-of-the-art DRL approaches, with competitive stability and computational efficiency. Evaluations with public datasets and real traffic data from Zhengzhou, Henan, China, confirm TCGP’s competitive performance and adaptability. This contributes fresh perspectives to the modernization of urban traffic systems.
1. Introduction
With urbanization accelerating, traffic congestion and delays increasingly impact city residents’ daily lives and quality of life. Recent reports indicate that, on average, drivers in London lost 156 h to congestion in 2022, a figure expected to rise as traffic volume increases due to factors such as population growth and urban expansion. In this scenario, traffic signal control (TSC) is essential for reducing congestion, improving traffic flow efficiency, and enhancing safety. The significant role of TSC in mitigating congestion involves advanced strategies like adaptive timing and coordinated signals. The growing interest in this research area has led scholars to develop various innovative methods and applications [1,2], including (1) traditional transportation engineering methods [3,4]; (2) metaheuristics methods [5,6]; and (3) deep reinforcement learning (DRL) methods [7,8,9].
TSC optimization is fundamentally an optimization challenge [7], escalating to an NP-complete problem as the scale of the road network expands [10]. In the field of TSC, fixed-time traffic signal strategies have dominated due to their simplicity, efficiency, and compatibility with existing rules. However, these traditional methods, often reliant on manual parameter setting, frequently fail to optimize results effectively. In contrast, recent advancements in DRL-based methods have garnered significant interest for their exceptional real-time performance and adaptability [11,12]. These methods have demonstrated superior outcomes in traffic simulations, often outperforming traditional approaches and achieving state-of-the-art results.
In practical applications, DRL methods still face several challenges: (1) Real-time data collection: DRL methods require extensive data, and the installation and maintenance of sensors, as well as data transmission and storage, not only increase the project cost but may also reduce practical feasibility and sustainability; (2) Adaptability and safety concerns in TSC: DRL approaches often lead to frequent changes in signal control methods, which can pose adaptability issues for traffic participants, thereby increasing the risk of traffic accidents; (3) Parameter sensitivity: DRL methods are highly sensitive to parameters and exhibit poor robustness in the face of varying traffic conditions, resulting in performance variability across different scenarios. Figure 1 shows the difference in phase sequencing between DRL and some traditional methods compared to our TCGP method. Figure 1a shows a possible execution process of other methods. The execution order of phases is unpredictable and may also cause a certain phase never to be executed. This paper explicitly frames the trade-off between adaptive responsiveness (favored by DRL) and operational stability (required by fixed-time systems). While DRL can adapt to real-time fluctuations, its unpredictable phase sequences often disrupt driver expectations, creating a significant ‘practical deployment gap’ in real cities. The TCGP method bridges this gap by providing a progressive optimization that maintains phase sequence stability while improving upon traditional fixed-time methods.
Figure 1.
Example of phase sequencing. (a) DRL methods may exhibit unpredictable phase jumps or skip phases entirely based on immediate rewards. (b) The TCGP method maintains a stable phase sequence, ensuring fairness and preventing phase starvation.
In light of these issues, this paper focuses on the widely used fixed-time traffic signal strategy in reality, exploring the optimization of fixed-time cycle lengths, and proposes a Traffic Characteristics-Guided Progressive Optimization (TCGP) method, which leverages traffic engineering characteristics to refine signal timings progressively. Figure 1b illustrates the phase sequencing of our method, which maintains phase continuity without sudden changes and enhances traffic flow continuity. Our TCGP framework is based on the hypothesis that optimizing a single intersection can benefit global optimization and convergence, with regional optimization being achieved through the successive optimization of individual intersections. Direct optimization of the traffic signal green ratio and signal cycle lengths at individual intersections uses prior knowledge from the traffic domain, simplifying the solution space. This makes the search process for traffic signal parameters more efficient and the results more stable. For regional road networks, we propose a progressive optimization strategy based on single-intersection optimizations. It integrates the proposed green time ratio and cycle length optimization methods into a progressive optimization framework, continuously selecting new intersections for optimization. The single optimization framework is shown in Figure 2. It shows the optimization process: adjusting the green light ratio of the phase according to the traffic conditions to ensure effective use of green light time. Cycle lengths are then explored, gradually adjusting to find the optimal duration.
Figure 2.
TCGP Framework: progressive optimization loop, green time ratio calculation based on traffic volume, and cycle length search via simulation.
Our method is different from other methods and has unique advantages. Compared to metaheuristics methods, the search space complexity of our proposed method does not exponentially increase with the number of intersections, enhancing the scalability. Additionally, our method achieves results comparable to those of DRL-based control methods. The optimized TSC schemes generated can be directly integrated into existing TSC systems, offering low deployment costs, high public acceptance, and validated effectiveness in the actual traffic network of Zhengzhou, Henan, China.
The main contributions of this study include the following:
- We designed a stepwise progressive optimization framework for intersections, applying our method in cyclic iterations to multiple intersections within the network to verify its convergence in multi-intersection scenarios.
- Integrating knowledge from characteristics of traffic engineering and turning ratios as prior guidance, we developed a formula based on traffic volume and turning ratios to quickly estimate the green time ratio at intersections, thus enhancing the efficiency of fixed-time traffic signal optimization.
- Considering the actual characteristics of intersections, we used the optimized green time ratio constraints to progressively explore and optimize the signal cycle at intersections, ensuring the reasonableness of the optimization results.
In practical applications, our method demonstrates unique advantages:
- This approach avoids exponential growth in dimensions as the number of intersections increases, thereby accelerating the method’s convergence.
- The TCGP method requires less real-time data and computing resources, allowing it to integrate into existing TSC systems and simplify deployment seamlessly.
- It reduces the instability caused by frequent changes in TSC modes, thus enhancing traffic flow safety.
- Its good scalability and reliability make it a better choice for optimizing large and complex traffic networks.
The remainder of this paper is organized as follows: Section 2 reviews recent developments in TSC optimization, outlining the characteristics, strengths, and limitations of existing methods, as well as the challenges they face. Section 3 provides a detailed description of our proposed method, including its core ideas, specific steps, and procedural flow. Section 4 discusses the simulation platform, comparative methods, experimental configurations, and evaluation metrics. Section 5 presents the experimental outcomes and comparative analyses, delving into the advantages of our method. Section 6 revisits the core contributions of the paper, systematically discusses the innovative aspects and unique advantages of our method, and suggests directions for future research.
2. Related Works
2.1. Traditional Transportation Engineering Methods
Traditional methods for TSC primarily derive from traffic flow and control theories proposed in the mid-20th century. These methods, usually based on vehicular traffic theory and combined with empirically measured traffic parameters, employ mathematical models and analytical methods to solve for optimal signal timing or rely on rule-driven control systems.
2.1.1. Formula-Based Fixed-Time Methods
In 1958, Webster introduced a fixed-time traffic signal optimization method known as the Webster formula method [13]. This approach calculates the cycle length and green time ratio for individual intersections based on traffic demand. The Webster formula is a classic example of a formula-based method widely used for its simplicity and ease of implementation.
2.1.2. Adaptive Control Systems
Following the development of fixed-time methods, adaptive systems like SCATS [14] and SCOOT [15] emerged. These systems adjust signal timing schemes in response to real-world conditions. SCATS uses a three-tier distributed structure to determine a common cycle length for subregions and selects appropriate timing schemes based on traffic flow at key intersections. SCOOT continuously fine-tunes timing parameters using a two-tier structure.
2.1.3. Rule-Driven Control Methods
Unlike methods based on traffic models, rule-driven control strategies operate according to predefined rule methods, adjusting traffic signals in response to traffic conditions. One example is the GreenWave method [16], which optimizes the phase offsets of traffic signals along a corridor to coordinate these timings and achieve a “green wave” effect. This ensures that vehicles traveling at the desired speed along the corridor do not encounter red lights at intersections, allowing for uninterrupted passage. Another example is Max-pressure control (MP) [3], originally designed for data scheduling in wireless communications and later adapted for TSC [17]. MP control uses only local traffic pressure information at intersections, calculating the pressure for each phase based on the differential vehicle count queued upstream and downstream. It then allocates green time to the phase with the highest pressure, lacking regional optimization capabilities. Self-Organizing Traffic Light Control (SOTL) [4] determines whether to maintain or switch the current phase based on observed traffic conditions and predefined rules.
2.2. Metaheuristics Methods
To solve TSC, many meta-heuristics are widely applied to minimize traffic delays by adjusting signal timing, cycles, phases, and offsets [6]. These intelligent methods utilize swarm intelligence or evolutionary methods to iteratively search for global optimal solutions. These methods are particularly effective for optimization problems involving a single intersection or limited parameters.
2.2.1. Evolutionary Algorithms
Evolutionary Algorithms, such as Genetic Algorithms (GA) and Differential Evolution (DE), have been applied to optimize signal timings. For instance, a multi-objective GA was used to maximize throughput and minimize delays [18]. DE was employed to optimize green time ratios and cycle lengths, showing significant improvements in delay reduction [19].
2.2.2. Swarm Intelligence Algorithms
Swarm Intelligence algorithms, including Particle Swarm Optimization (PSO) and Ant Colony Optimization, have also been used extensively. PSO has been applied to optimize signal timings and offsets and demonstrated reductions in average vehicle delays [20]. Another study [21] used an improved rank-based ant system algorithm, enhancing the algorithm by incorporating local search techniques.
2.3. Deep Reinforcement Learning Methods
With the development of deep neural networks, DRL has shown outstanding performance in optimizing TSC. Reinforcement learning can be categorized into value learning, policy learning, and algorithms that combine both. In the context of DRL, TSC actions primarily involve selecting a specific signal phase. Thus, methods based on value functions [22,23] are more widely applied, with the DQN algorithm being particularly representative. DRL-based TSC methods can be divided into two main categories: global single-agent control and multi-agent control.
2.3.1. Global Single-Agent Control
Global single agent control offers a straightforward solution where a central agent receives global traffic state information and learns to set signal actions for all intersections [24]. However, as the number of intersections increases, this method becomes prone to the “curse of dimensionality,” making it challenging to associate reward signals with specific actions and learn stable strategies.
2.3.2. Multi-Agent Control
The use of multi-agent algorithms has become mainstream, with each intersection assigned an agent [25]. IntelliLight [26] proposed a specific DRL algorithm, showing its effectiveness in reducing traffic congestion through simulations. CoLight [27] introduces communication to coordinate actions between intersections. FRAP [28] proposes a flexible network that can be used at different intersections by flipping and rotating. On the basis of FRAP, the Advanced-XLight algorithm [12] uses an improved advanced pressure model as the DRL state. This approach enables DRL to learn the threshold for pressure switching quickly and to converge more easily, achieving state-of-the-art results. Additionally, some research is focused on applying DRL algorithms in scenarios where data from some intersections is missing [29], aiming to expedite the deployment of DRL algorithms.
2.4. Further Analysis on Related Works
After reviewing the three main methods of TSC, we make a further analysis of the advantages and disadvantages of these three methods.
2.4.1. Analysis of Traditional Transportation Engineering Methods
Traditional methods, including formula-based fixed-time, adaptive control systems, and rule-driven control strategies, are well-established and easy to implement. However, they still have some limitations:
- Lack of Adaptability: Fixed-time methods cannot adjust to traffic conditions with different congestion levels, resulting in poor performance under fluctuating conditions.
- High Computational Complexity: Adaptive systems like SCATS and SCOOT require extensive real-time data and have high computational demands.
- Localized Optimization: Rule-driven methods, such as maximum pressure control and SOTL, do not consider global traffic information and lead to frequent intersection phase switching.
2.4.2. Analysis of Metaheuristic-Based Methods
These methods can efficiently search the solution space of signal timing and provide good solutions for single intersections or scenarios with limited parameters. However, they also have notable drawbacks (Scalability Issues): As the number of intersections increases, the solution space grows exponentially, making it challenging to maintain optimization efficiency.
2.4.3. Analysis of Deep Reinforcement Learning Methods
DRL has shown promise in achieving high performance in TSC by leveraging deep neural networks. DRL methods are particularly good at handling dynamically changing traffic conditions by constantly changing actions. Although TSC based on DRL is currently a popular research direction, its full adoption in real-world settings is challenging due to several factors.
- Simulation Dependency: Training in simulated environments may not accurately capture real-world traffic complexities, leading to instability when deployed in actual scenarios.
- Action Variability: DRL methods may cause unpredictable phase changes, making it difficult for traffic participants to adapt, and may also lead to safety accidents.
- Robustness Issues: DRL models can be sensitive to anomalous data, resulting in poor robustness as shown in studies [30,31].
Recent works, such as joint control of phase sequence and timing [32], demonstrate the potential of flexible phase design. However, such methods significantly increase control complexity and may not be compatible with legacy fixed-time infrastructure. Similarly, while connected-vehicle applications [33] focus on safety warnings, TCGP method addresses safety at the control source by ensuring phase sequence stability, thereby reducing the risk of sudden stops and red-light violations caused by erratic DRL policies.
2.4.4. Our Proposed Method
To address the limitations of the above methods, we propose the TCGP method. TCGP method integrates traffic engineering characteristics with a progressive optimization strategy, offering several advantages:
- Enhanced Adaptability: By using stepwise optimization, TCGP method can adapt to traffic variations more effectively than traditional fixed-time methods.
- Reduced Computational Complexity: Historical traffic data is used to calculate the green time ratio of traffic lights, reducing computational requirements compared to metaheuristic methods.
- Global Optimization: TCGP method considers the global traffic flow status and uses global indicators as optimization goals to solve the limitations of rule-based methods.
- Practical Deployment: TCGP method facilitates direct integration into existing traffic control systems without changing past traffic habits, solving practical deployment challenges in the DRL approach.
3. The Proposed Method
3.1. Problem Formulation
To facilitate a clear understanding of the TCGP method, we first define the key notations and concepts used throughout this article. Table 1 summarizes the notations, and Figure 3 shows an intersection with four-phase control.
Table 1.
Notations used in this article.
Figure 3.
Intersection and phase introduction.
TSC is an important aspect of urban traffic management that aims to minimize congestion and improve traffic flow efficiency. We can abstract it as an optimization problem. Optimizing TSC timing can help improve urban traffic congestion. The purpose of this study is to optimize the signal timing at intersections to minimize the average travel time of vehicles and thereby improve traffic efficiency. Additionally, our method ensures compatibility with existing control systems. Our method is guided by traffic engineering characteristics. Unlike generic coordinate descent, TCGP utilizes the explicit relationship between traffic volume and green time (Equation (7)) to guide the search, ensuring physical interpretability. It seeks to optimize the traffic signal timings by adjusting the and C at each intersection (J) while considering the effective number of lanes () and traffic volumes of phases (). The key concepts used in our method are defined as follows:
Green Time Ratio. The green time ratio is the ratio of a traffic signal phase’s effective green light duration to the total cycle length. The green time ratio for phase can be expressed as:
Effective Number of Lanes for Phase. The effective number of lanes () for phase i represents the adjusted count of lanes considering mixed traffic flows. If a lane allows multiple movements (e.g., left turns, straight-through, right turns), a mixed ratio () is used. For example, if a lane permits three movements equally, , and the lane contributes to the effective number of lanes for that phase. The effective number of lanes for phase i is:
where is the mixed ratio for lane j in phase i.
Effective Traffic Volume for Phase. The effective traffic volume () for phase i represents the number of vehicles passing through the intersection during a time period t, controlled by phase i. It can be expressed as:
where is the number of lanes for lane j during phase i, and is the vehicle flow rate (vehicles per unit time) for lane j during phase i.
3.2. Overall Framework
In order to achieve the optimization of the regional road network, we propose a novel TCGP method. This method concentrates on green time ratio optimization and cycle length optimization in a regional network progressive framework. Figure 4 clearly describes the structure and content of the entire method. The objective function of the proposed TCGP is the average time loss, which has a certain relationship with the average travel time. The specific definitions of the two are as follows:
Figure 4.
Algorithm framework diagram. (a) represents a progressive optimization framework for the regional network, while (b,c) are optimizations for individual intersections, focusing on green time ratio and cycle lengths.
Average Travel Time. Average travel time () represents the average duration that all vehicles take to travel within the road network. It includes both the time spent traveling and the time spent stopped or waiting. The average travel time is given by:
Average Time Loss. Average time loss () represents the average time lost by vehicles traveling below the desired speed (), including waiting time. The total travel time () is the sum of the time vehicles would travel at the desired speed and the average time loss. The relationship is given by:
The objective function of the TCGP:
We choose average time loss as the objective function because it directly measures the efficiency loss of vehicles in the traffic network. Average Time Loss can reflect bottlenecks and inefficient areas in actual traffic conditions. By minimizing average time loss, it can effectively reduce vehicle stagnation time and low-speed driving time and improve overall traffic smoothness.
In order to optimize this problem, our method proposes three modules for hierarchical and progressive optimization. Here, we briefly introduce these three modules. Figure 4 clearly describes the overall framework and the three modules.
- Regional Network Progressive Optimization: Figure 4a represents regional progressive search optimization aimed at effectively circumventing the “combinatorial explosion” challenge posed by optimizing the signal timing parameters of all intersections simultaneously on a global scale. The approach fixes all intersections’ cycle lengths except one to their current optimal values (indicated in red). Then, the cycle lengths of the selected single intersection are optimized to achieve a local optimum under the current network conditions. The intersection being optimized is marked with a green circle in the diagram. This process is repeated in a cycle until the cycle lengths of all intersections in the network converge to the optimum.
- Green Time Ratio Optimization: In Figure 4b represents the optimization of the green time ratio for a single intersection. For the selected intersection, the algorithm initially calculates the optimal green time ratio configuration for each phase based on the traffic volume of each approach. It demonstrates the method for calculating the traffic volume and the green time ratio at a typical four-phase intersection, linking actual traffic volumes to the theoretically optimal green time ratios.
- Cycle Length Optimization: Figure 4c is about the cycle length optimization for a single intersection. After determining the optimal green time ratio, the algorithm searches for the best cycle length for that intersection through traffic simulation. The diagram shows the process of iterating through different candidate cycle lengths and evaluating performance within a traffic simulation environment.
The three modules are iteratively interconnected through Figure 4a until the signal timing of all intersections in the network converges to the global optimum. This process cleverly decomposes the global optimization problem into a series of local optimization sub-problems. In the following sections, a detailed introduction to these three modules will be provided.
3.3. Optimization
3.3.1. Regional Network Progressive Optimization
This paper introduces an innovative Regional Network Progressive Optimization strategy aimed at breaking through the bottlenecks of regional optimization. Our starting point is the recognition that changing the signal timing at any single intersection will affect the overall traffic efficiency of the entire area. Based on this, optimizing the traffic timing parameters of all intersections within a region simultaneously would lead to a “combinatorial explosion” of the search space, posing a significant computational challenge for optimization. To effectively circumvent this issue, we adopted a “divide and conquer” strategy, breaking down the regional optimization into a series of single intersection optimization sub-problems.
- Fix Traffic Signal Timing for All Other Intersections: Initially, the signal timings for all intersections except one are held constant at their current optimized values.
- Optimize Signal Timing for a Single Intersection: Select an intersection and optimize its signal timing based on the current network state, aiming to find the local optimum for that intersection.
- Iterative Process for All Intersections: This progressive strategy is distinct from isolated optimization. Although one intersection is optimized at a time, the updated signal timing of the optimized intersection becomes part of the environment for the next iteration. This iterative feedback mechanism allows the optimization to propagate through the network, capturing the interactions between intersections without the computational burden of global simultaneous optimization.
After one round, the overall effect for the entire area may not be optimal, necessitating several rounds of repetition of the above process for the entire area until the global indicators converge, as shown in Figure 4a. The algorithm process will be described in Algorithm 1 later.
The advantage of the Regional Network Progressive Optimization strategy lies in its focus on optimizing individual intersections, significantly reducing search complexity, and maintaining efficiency in large-scale road networks while ensuring solution precision. Compared to directly optimizing the entire area globally, this progressive framework transforms a high-dimensional global optimization problem into multiple low-dimensional local optimization sub-problems, avoiding the “curse of dimensionality.” Furthermore, our method can treat multiple intersections as a sub-area, defining the content of the sub-area as a minimum unit and progressively optimizing it, thereby enabling optimization of larger road network areas.
| Algorithm 1: Progressive optimization of traffic signals across multiple intersections in a region |
|
The TCGP method utilizes a progressive optimization strategy to address the computational challenges of optimizing large traffic networks. Instead of optimizing the signal timings of all intersections simultaneously, which would lead to an intractable increase in search space, TCGP method fixes the signal timings of all intersections except one at each step. The selected intersection is then optimized for its green time ratio and cycle length based on the current network conditions. This process is repeated for all intersections in the network, ensuring that the optimization is both manageable and computationally efficient. To determine the order of intersection optimization, intersections with the highest traffic pressures or those that cause significant network bottlenecks are prioritized. This ensures that the global traffic efficiency is improved progressively, while minimizing the risk of suboptimal configurations.
3.3.2. Green Time Ratio Optimization
The Regional Network Progressive Optimization discussed in the previous subsection addresses the optimization of multiple regional intersections. This subsection focuses on optimizing individual intersections, breaking down the optimization into green time ratio optimization and cycle length optimization for a single intersection. This approach simplifies the number of parameters involved. We adopt an innovative method based on traffic engineering principles, which calculates the optimal green time ratio configuration for each phase directly from traffic volume data. The core goal of TSC is to reasonably allocate green light resources at intersections based on actual traffic volumes. Traditional methods often start with an assumed initial green time ratio configuration, then continuously adjust it based on simulation feedback—a process akin to “Blind Men and an Elephant,” which introduces inefficiencies and uncertainties. Our method is based on the following considerations:
- The green time ratio of a phase should be closely related to the number of lanes it controls and the actual traffic volume.
- For most intersections, right-turn traffic flows are not controlled by signals and can be considered as having a constant green phase.
- Within a given statistical period, the traffic volume that each phase can accommodate is directly proportional to its green time ratio.
Based on the principles mentioned, we propose a novel formula for calculating the green time ratio: for each phase , we first calculate the effective number of lanes it controls and then tally the effective traffic volume controlled by that phase during a specified time. Next, we calculate the “relative traffic volume” for each phase as . Finally, through normalization, the green time ratio for each phase is directly proportional to its share of the relative traffic volume. The specific formula is presented as Equation (7).
The rationale behind this formula is that the green time ratio should be proportional to the effective traffic volume and lane capacity, ensuring that phases with higher demand receive a larger share of green time. This formula is grounded in basic traffic flow theory, where each phase’s green time is proportional to the volume of vehicles it needs to serve, adjusted by the effective number of lanes.
This formula calculates the green time ratio directly from actual traffic flow data, thus eliminating the need for manual presetting of values. An example calculation is presented in Figure 4b, while the algorithmic process for calculating the green time ratio at all intersections is detailed in Algorithm 2. Moreover, because the calculation involves only basic operations on statistical data, its computational complexity is extremely low, making it suitable for large-scale applications. Extensive practical testing has demonstrated that the green time ratio configurations calculated by this method often yield excellent traffic throughput effects. Applying this formula to each phase of the traffic lights across the network can quickly optimize the green time ratios at all intersections. Thus, the innovative design of Figure 4b is key to ensuring the efficiency and precision of the entire algorithm framework.
| Algorithm 2: Optimization of green time ratio at a single intersection |
|
3.3.3. Cycle Length Optimization
Cycle length optimization at intersections is a crucial step in enhancing overall performance, building upon green time ratio optimization. We have implemented an efficient heuristic search strategy that quickly determines the optimal cycle length configuration for each intersection within a few iterations. Cycle length is another critical parameter in signal timing, directly influencing the intersection’s capacity and queue length. However, merely extending the cycle length may not necessarily enhance efficiency due to green time ratio constraints; thus, determining the optimal cycle length is a complex matter. Our approach is as follows:
- For the selected intersection, iterate over candidate cycle length values within a reasonable interval.
- Combining the cycle length with the calculated optimal green time ratio, the complete traffic signal timing plan for the intersection is calculated using Equation (8).
- Apply this traffic signal timing scheme to a traffic simulation model to obtain corresponding network performance indicators, with this study focusing on average time loss.
- Among all candidate cycle lengths, select the one that optimizes network performance as the optimized cycle length for the intersection. The final cycle length determination is further refined through the Regional Network Progressive Optimization described in Section 3.3.1.
In cycle length optimization, we utilize advanced simulation models that precisely assess network performance under various timing parameters, as detailed in Algorithm 3. While a grid search is employed, the search space for cycle length is typically limited (e.g., 30–120 s). Combined with our linear complexity analysis, this ensures the method remains computationally efficient for large-scale networks.
In summary, by integrating this module with the first two, we efficiently calculate the optimal green time ratio and cycle length for each intersection. Algorithm 1 details the entire process of the TCGP, illustrating how it innovatively addresses traffic signal timing optimization.
Novel Aspects of the TCGP Method: The Traffic Characteristics-Guided Progressive Optimization (TCGP) method offers several novel contributions to the field of traffic signal control. Firstly, it introduces an innovative progressive optimization strategy that decomposes large-scale regional optimization into a series of local optimization sub-problems, mitigating the “curse of dimensionality” commonly encountered in global optimization approaches. Secondly, the TCGP method integrates traffic engineering knowledge into the optimization process, specifically utilizing traffic volume and effective lane counts to calculate the green time ratio, rather than relying solely on empirical rules or simulation feedback. Finally, TCGP method ensures smooth integration with existing fixed-time signal systems, offering a practical, cost-effective solution for real-world traffic networks.
3.4. Complexity Analysis
The proposed method comprises three main components; we provide a complexity analysis for the entire algorithm.
3.4.1. Complexity of Regional Network Progressive Optimization
Let N be the number of intersections in the network and I be the number of iterations required for convergence. The time complexity for this component can be expressed as:
where is the time complexity for optimizing a single intersection. It is dominated by the complexity of Cycle Length Optimization for a single intersection.
3.4.2. Complexity of Green Time Ratio Optimization
The time complexity for calculating the green time ratio for each intersection is determined, and the total complexity for all intersections is .
3.4.3. Complexity of Cycle Length Optimization
Let C be the number of candidate cycle lengths. For each cycle length, the traffic simulation must be run to evaluate performance. is the time of running the traffic simulation, it is a constant. The time complexity for this component for a single intersection is .
3.4.4. Overall Complexity
Combining the complexities of the three components, the overall time complexity of the proposed method is:
This complexity indicates that the proposed method scales linearly with the number of intersections, and it also depends on the number of candidate cycle lengths and the complexity of the traffic simulation.
| Algorithm 3: Cycle length optimization at an intersection for traffic signal timing |
|
4. Experiment
In this section, we detail the simulation experiments conducted to validate the method we proposed. We performed validations using several public datasets as well as real traffic data from Zhengzhou, China. Our results were compared with those from methods provided by the open-source LibSignal(v1.0.0) [34] to assess our method’s performance comprehensively. This section provides a detailed introduction from various perspectives, including experimental setup, datasets used, selection of comparison methods, and criteria for evaluation metrics. These details offer readers a clear and comprehensive experimental background. Through validations across multiple datasets and various scenarios, we aim to verify the robustness and applicability of our method, thereby gaining a deeper understanding of its performance in real traffic scenarios.
4.1. Experiment Settings
In this experiment, we used SUMO Simulation of Urban Mobility (SUMO, version 1.18.0) [35], an open-source and highly portable micro-level continuous traffic flow simulation software designed for large-scale networks. SUMO is known for its high precision and reliability. To comprehensively evaluate the performance of the method proposed in this paper, our simulation results are thoroughly compared with those from methods available in the LibSignal library [34], a benchmark resource for TSC tasks that includes a variety of classic optimization methods.
In our experiments, we followed the setup used in LibSignal. This setup includes a 5-s yellow light phase as a buffer after the green phase. To address the differences in various intersections, we mainly used the common 4-phase setting found in public datasets. For real urban network data, we used the actual phase configurations used locally to simulate real situations better. This approach ensures consistency and reproducibility of the experiments. It also closely mimics real-world traffic conditions, helping objectively evaluate the method’s performance in practical applications.
4.2. Datasets
In our experiments, we carefully selected five road network datasets and nine traffic flow datasets to ensure the comprehensiveness of the experiments. The public road network datasets [34] come from cities like Hangzhou and Cologne, as shown in Figure 5, covering different cities and road network topologies. Our Zhengzhou road network data realistically reflects the local urban roads and traffic conditions, as shown in Figure 6.
Figure 5.
Public datasets road network. It shows the road network situation in the public data set used, covering most situations, including a single intersection road network, a trunk-coordinated road network, and a regional road network.
Figure 6.
Zhengzhou datasets road network. Contains two real road networks in Zhengzhou, each with three different levels of traffic flow. Covers a variety of intersection situations.
Each public road network dataset is paired with a corresponding traffic flow dataset, while the two Zhengzhou road network datasets are provided with three sets of real traffic data representing different congestion levels (mild, moderate, severe). By utilizing a diverse range of road network topologies and traffic data from multiple sources, we aim to comprehensively evaluate the performance of the proposed method under various complex scenarios, verifying its general applicability and robustness.
- HangZhou1x1 Road Network [36]: Illustrated in Figure 5a, this public dataset originates from camera data at an intersection in Hangzhou. Each direction of this intersection is equipped with one lane for left turns and one for straight-ahead traffic, excluding right turns from the calculation.
- HangZhou4x4 Road Network [27,28]: Shown in Figure 5b, this network encompasses 16 intersections. Each intersection has four incoming and four outgoing roads, each featuring three lanes designated for left turns, straight-ahead traffic, and right turns. Based on camera data from Hangzhou, the traffic flow data has been simplified for analysis.
- Cologne1x3 Road Network [37]: Displayed in Figure 5c, this complex network in Cologne features irregular intersections, with only three of them being signalized. The remaining intersections are not controlled by traffic lights, and the phase configurations do not fully comply with standard phasing.
- ZhengZhou-15 Road Network: Presented in Figure 6a, we collected this network, which consists of 15 signalized intersections with heterogeneous lane numbers and distinct phase settings among the intersections. Based on this network, we collected three traffic data sets representing severe, moderate, and mild traffic saturation levels. The traffic flow data, derived from intersection camera footage, includes various turning ratios, which are then used to model traffic flows. Unlike other public traffic datasets, vehicles in this dataset may originate from within the network, not just from its perimeter, more closely resembling real-life conditions.
- ZhengZhou-39 Road Network: Shown in Figure 6b, this dataset also covers an area in Zhengzhou, including 39 intersections. The setup of intersections and traffic volume data is similar to that of the ZhengZhou-15 dataset.
4.3. Compared Methods
To comprehensively evaluate the performance of our method, this experiment compares the results with various classic methods from the authoritative LibSignal library. For public datasets, we use the existing experimental results from the LibSignal library as a baseline for comparison. For the self-collected real road network data from Zhengzhou, we have implemented a series of classic methods from LibSignal to conduct a more thorough and comprehensive comparative analysis. Each method includes a 5-s yellow light phase for clearing the intersection at phase transitions. The baseline methods used in the Zhengzhou dataset are as follows:
- FixedTime [13]: A predetermined fixed phase timing scheme that does not consider actual traffic conditions. We will compare the performance with each phase set to 10 s and 30 s timing.
- MaxPressure [3]: Calculates the pressure at an intersection and chooses the phase with the highest pressure for green light clearance.
- SOTL [4]: The SOTL method adjusts the signal phase by considering competing phases and determining the next phase based on traffic flow and preset rules.
- IDQN [34]: A TSC method based on reinforcement learning, using the Deep Q-Network (DQN) algorithm from DRL to determine the signal action at a single intersection based on its current state.
- PressLight [38]: A TSC method based on DRL, which utilizes the concept of maximum pressure. It allows the reinforcement learning agent to learn the state and thresholds of pressure to control phases.
The methods mentioned above are classic representations in the field of signal control. They encompass a variety of optimization paradigms, including fixed rules and reinforcement learning. They effectively reflect the highest level of existing methods.
4.4. Evaluation Metrics
To evaluate our TCGP method, we adopted evaluation metrics consistent with those in the LibSignal [34] library, covering several commonly used evaluation methods in the field of traffic signal optimization. These metrics include average travel time, queue length, delay, and throughput, providing a comprehensive reflection of the effectiveness and performance of TSC. For convenience of comparison, we calculated the indicator similarly to Libsignal. However, we believe that average travel time and throughput are more important because the other two indicators are due to average calculation problems, and we think that the original calculation method is flawed. Here, we also list them for reference.
- Average Travel Time: This refers to the average time each vehicle spends in the traffic network, including both waiting and actual travel times. A lower average travel time indicates a higher efficiency of the traffic system. This metric is widely considered the primary criterion in signal optimization and is a key comparative indicator in this study.
- Queue Length: This refers to the average number of vehicles waiting in each lane during a certain period. A smaller queue length indicates lower traffic congestion and higher intersection space utilization.
- Delay: Defined as the average difference between the current speed of all vehicles and the maximum speed limit of their lanes. A smaller delay time indicates better performance.
- Throughput: The number of vehicles that complete their journeys during the entire simulation process. Higher throughput indicates a stronger traffic network capacity and a higher service level.
The above metrics quantitatively evaluate the method from multiple dimensions, including time efficiency, space utilization, smoothness of traffic flow, and overall traffic capacity, demonstrating strong comprehensiveness and representativeness.
5. Result
In this section, we will compare the experimental results of our method with those of the benchmark methods, dividing the comparison into two parts: one for the public datasets and the other for the Zhengzhou dataset. To examine the performance of our method under different traffic volumes, we divided the Zhengzhou dataset into three saturation levels (mild, moderate, and severe) and conducted experiments for each level separately.
5.1. Public Dataset Results
This section compares the experimental results from three open-source traffic datasets: HangZhou1x1, HangZhou4x4, and Cologne1x3. Additionally, we will list the results of the benchmark methods, with their data sourced from the LibSignal library. Table 2 shows the results on three public datasets. The best result is marked in red, the second-best result in blue, and our results are shown in bold. From the data of a single intersection of HangZhou1x1, our method has already achieved performance close to the optimal level in terms of average travel time, with only a small difference from the best result. In the dataset of HangZhou4x4, our method can reach the mainstream level. Furthermore, in the Cologne1x3 dataset, we achieved the best results.
Table 2.
Results from public datasets. Three data sets are represented: HangZhou1x1, HangZhou4x4, and Cologne1x3.
Compared with other fixed-time methods and rule-driven methods, our method can achieve the best results. Compared with the DRL-based methods, our method can surpass most other methods when it is more suitable for reality. The results that surpass our method are often obtained using methods that employ non-fixed phase sequences with random actions. However, such random methods may face various challenges in practical applications. In contrast, our method demonstrates greater reliability and practicality in real-world scenarios. Next, we will present the results of our method on the Zhengzhou dataset.
5.2. Results Under Datasets of Varying Traffic Congestion Levels
For each Zhengzhou road network dataset, covering three different levels of traffic saturation (mild, moderate, severe), we divided these data into two main groups for an in-depth comparison. Table 3 showcases the performance of the ZhengZhou-15 network under different traffic scenarios, including mild, moderate, and severe congestion. Table 4 shows the performance of our method within the ZhengZhou-39 network. These tables provide detailed insights into the adaptability of our method in traffic scenarios with varying degrees of congestion.
Table 3.
Results for different congestion levels in ZhengZhou-15 road network.
Table 4.
Results for different congestion levels in ZhengZhou-39 road network.
In the two Zhengzhou network datasets (ZhengZhou-15, ZhengZhou-39), our method also performs better compared to some methods and significantly outperforms manually optimized timing schemes used in practice. In this comparison, we only compared two main DRL-based methods. This is because many methods use the “symmetry invariance” framework, which makes them unable to be used in complex road networks without processing. Compared with the DRL-based methods, in the ZhengZhou-15 data, our method can achieve a close level under the periodic loop control method. We noticed that in the ZhengZhou-39 data, the throughput of the DRL-based methods becomes smaller with the degree of congestion. This is because the DRL-based methods sacrifice the passability of some intersections.
Notably, from light to heavy congestion, the performance ranking of some methods shows significant differences, while our method maintains relatively stable optimization effects. Especially when the saturation reaches heavy congestion, DRL-based methods face issues due to multiple agents competing without reaching a Nash equilibrium, causing the final results to fluctuate. Furthermore, the timing optimized by our method has been validated in some actual areas of Zhengzhou city, with the optimization results surpassing those of the original manually optimized timings.
The TCGP algorithm cannot always be in the leading position in the above results. However, it is worth noting that our algorithm is based on the optimization of fixed-time traffic signal. Compared with phase random algorithms, our algorithm has more practical value, which is also an advantage.
5.3. Convergence Analysis
In this subsection, we will focus on analyzing the convergence behaviour of the method during the optimization process. Figure 7 and Figure 8 display the convergence curves of the method on the Zhengzhou 15-intersection network and the Zhengzhou 39-intersection network, respectively. The x-axis represents the number of optimization rounds, and since our method’s objective function is the average loss time, the average loss time is plotted on the y-axis. At the same time, we have also compared several DRL methods. Since these DRL methods use the average travel time as a basis for observation, the average travel time is used as the y-axis for these methods.
Figure 7.
Convergence of various congestion levels in ZhengZhou-15. The table below shows our convergence curve. Each epoch indicates that the simulation runs for a round, and the minimum indicator of this round is the result.
Figure 8.
Convergence of various congestion levels in ZhengZhou-39. The table below shows our convergence curve. Each epoch indicates that the simulation runs for a round, and the minimum indicator of this round is the result.
The graphs clearly show that, under conditions of light, moderate, or heavy congestion, our method is able to rapidly converge to a near-optimal value of the objective function within a limited number of iterations. Even in more complex network settings, the method achieves desirable convergence outcomes within a reasonable number of iterations. Moreover, our method exhibits better stability. Under high saturation traffic conditions, methods based on DRL show significant fluctuations and struggle to achieve a Nash equilibrium among multiple agents. However, our method maintains a good level of performance stability.
This outstanding convergence performance benefits from the progressive and mutually reinforcing optimization strategies within our method framework. Specifically, we start by determining a reasonable initial timing at the single-intersection level, followed by gradual global tuning at the network level. This point-to-surface paradigm enables the method to quickly approximate the global optimum across the entire solution space, avoiding the common pitfall of local minima traps seen in traditional methods.
Moreover, it’s noteworthy that even under extreme conditions of heavy congestion, the method’s convergence curve remains relatively smooth, without any severe oscillations or divergent behaviour. This fully demonstrates the robustness and resilience of our method framework, which can adapt to complex and variable real-world traffic scenarios.
Overall, our method displays exceptional convergence performance, not only achieving excellent near-optimal solutions rapidly but also exhibiting outstanding convergence stability under various congestion levels. This lays a solid foundation for the practical application of the method in real-world scenarios.
6. Conclusions and Future Work
This paper presents the Traffic Characteristics-Guided Progressive Optimization (TCGP) method, which offers a novel approach to optimizing traffic signal timings in urban road networks. By integrating traffic engineering characteristics and adopting a progressive optimization strategy, TCGP method effectively addresses the challenges posed by large-scale network optimization, ensuring both computational efficiency and practical applicability. Experimental results demonstrate that TCGP method achieves competitive performance comparable to state-of-the-art DRL methods, while offering distinct advantages in terms of stability and computational efficiency. Our method’s primary advantage lies in its practical deployability within existing fixed-time control systems, ensuring compatibility and safety. Future work will focus on enhancing rapid response capabilities.
The TCGP method is highly adaptable and compatible, allowing it to integrate seamlessly with other optimization methods. For example, it can accommodate green wave corridors by fixing their parameters and focusing on optimizing the remaining sections. Future research will focus on enhancing the method’s rapid response to sudden, abnormal traffic volumes by constructing a library of optimized timing plans for various scenarios. Continuous input of real-time traffic data will enable dynamic matching and output of suitable signal control plans, facilitating autonomous, around-the-clock scheduling strategies. These enhancements aim to make TCGP method more practical and effective in real time, contributing to the intelligent, automated development of urban traffic management systems and driving urban traffic towards smarter, greener, and more efficient in the future.
Author Contributions
Conceptualization, Y.W. and Z.Z.; Methodology, Y.W. and H.G.; Software, H.G. and Y.H.; Validation, Y.H.; Formal analysis, Y.H.; Investigation, H.G.; Resources, Y.W.; Data curation, H.G.; Writing—original draft, H.G.; Writing—review & editing, Y.W. and Z.Z.; Visualization, Y.H.; Supervision, Y.W. and Z.Z.; Project administration, Y.W.; Funding acquisition, Y.W. All authors have read and agreed to the published version of the manuscript.
Funding
This work was funded by the National Natural Science Foundation of China under grant 62002330.
Data Availability Statement
The original contributions presented in this study are included in the article. Further inquiries can be directed to the corresponding author.
Conflicts of Interest
The author Yuanhao Hu comes from China Mobile Group Henan Co., Ltd. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
References
- Ye, B.L.; Wu, W.; Ruan, K.; Li, L.; Chen, T.; Gao, H.; Chen, Y. A survey of model predictive control methods for traffic signal control. IEEE/CAA J. Autom. Sin. 2019, 6, 623–640. [Google Scholar] [CrossRef] [Scilit]
- Haydari, A.; Yılmaz, Y. Deep Reinforcement Learning for Intelligent Transportation Systems: A Survey. IEEE Trans. Intell. Transp. Syst. 2022, 23, 11–32. [Google Scholar] [CrossRef] [Scilit]
- Varaiya, P. Max pressure control of a network of signalized intersections. Transp. Res. Part C Emerg. Technol. 2013, 36, 177–195. [Google Scholar] [CrossRef] [Scilit]
- Cools, S.B.; Gershenson, C.; D’Hooghe, B. Self-organizing traffic lights: A realistic simulation. In Advances in Applied Self-Organizing Systems; Springer: London, UK, 2013; pp. 45–55. [Google Scholar]
- Shaikh, P.W.; El-Abd, M.; Khanafer, M.; Gao, K. A Review on Swarm Intelligence and Evolutionary Algorithms for Solving the Traffic Signal Control Problem. IEEE Trans. Intell. Transp. Syst. 2022, 23, 48–63. [Google Scholar] [CrossRef] [Scilit]
- Srivastava, S.; Sahana, S.K. A survey on traffic optimization problem using biologically inspired techniques. Nat. Comput. 2020, 19, 647–661. [Google Scholar]
- Wei, H.; Zheng, G.; Gayah, V.; Li, Z. Recent advances in reinforcement learning for traffic signal control: A survey of models and evaluation. ACM SIGKDD Explor. Newsl. 2021, 22, 12–18. [Google Scholar]
- Yau, K.L.A.; Qadir, J.; Khoo, H.L.; Ling, M.H.; Komisarczuk, P. A survey on reinforcement learning models and algorithms for traffic signal control. ACM Comput. Surv. 2017, 50, 1–38. [Google Scholar] [CrossRef] [Scilit]
- Mannion, P.; Duggan, J.; Howley, E. An experimental review of reinforcement learning algorithms for adaptive traffic signal control. In Autonomic Road Transport Support Systems; Birkhauser: Cham, Switzerland, 2016; pp. 47–66. [Google Scholar]
- Long, G.; Wang, A.; Jiang, T. Traffic Signal Self-Organizing Control with Road Capacity Constraints. IEEE Trans. Intell. Transp. Syst. 2022, 23, 18502–18511. [Google Scholar] [CrossRef] [Scilit]
- Arulkumaran, K.; Deisenroth, M.P.; Brundage, M.; Bharath, A.A. Deep Reinforcement Learning: A Brief Survey. IEEE Signal Process. Mag. 2017, 34, 26–38. [Google Scholar] [CrossRef] [Scilit]
- Zhang, L.; Wu, Q.; Shen, J.; Lü, L.; Du, B.; Wu, J. Expression might be enough: Representing pressure and demand for reinforcement learning based traffic signal control. In Proceedings of the International Conference on Machine Learning, Baltimore, MD, USA, 17–23 July 2022; pp. 26645–26654. [Google Scholar]
- Koonce, P.; Rodegerdts, L.; Lee, K.; Quayle, S.; Beaird, S.; Braud, C.; Bonneson, J.; Tarnoff, P.; Urbanik, T. Traffic signal timing manual. In Technical Report United States; Federal Highway Administration: Washington, DC, USA, 2008. [Google Scholar]
- Lowrie, P. Scats-a traffic responsive method of controlling urban traffic. In Sales Information Brochure Published by Roads & Traffic Authority, Sydney, Australia; UC Berkeley: Berkeley, CA, USA, 1990. [Google Scholar]
- Hunt, P.; Robertson, D.; Bretherton, R.; Royle, M.C. The SCOOT on-line traffic signal optimisation technique. Traffic Eng. Control 1982, 23, 190–192. [Google Scholar]
- Ye, B.L.; Wu, W.; Zhou, X.; Mao, W.; Huang, Y.S. A green wave band based method for urban arterial signal control. In Proceedings of the 11th IEEE International Conference on Networking, Sensing and Control, Miami, FL, USA, 7–9 April 2014; pp. 126–131. [Google Scholar] [CrossRef] [Scilit]
- Tassiulas, L.; Ephremides, A. Stability properties of constrained queueing systems and scheduling policies for maximum throughput in multihop radio networks. In Proceedings of the IEEE 29th IEEE Conference on Decision and Control, Honolulu, HI, USA, 5–7 December 1990; Volume 4, pp. 2130–2132. [Google Scholar] [CrossRef] [Scilit]
- Li, X.; Sun, J.Q. Signal Multiobjective Optimization for Urban Traffic Network. IEEE Trans. Intell. Transp. Syst. 2018, 19, 3529–3537. [Google Scholar] [CrossRef] [Scilit]
- Cakici, Z.; Murat, Y.S. A differential evolution algorithm-based traffic control model for signalized intersections. Adv. Civ. Eng. 2019, 2019, 7360939. [Google Scholar] [CrossRef] [Scilit]
- García-Nieto, J.; Alba, E.; Olivera, A.C. Swarm intelligence for traffic light scheduling: Application to real urban areas. Eng. Appl. Artif. Intell. 2012, 25, 274–283. [Google Scholar] [CrossRef] [Scilit]
- Renfrew, D.; Yu, X.H. Traffic signal optimization using Ant Colony Algorithm. In Proceedings of the 2012 International Joint Conference on Neural Networks (IJCNN), Brisbane, Australia, 10–15 June 2012; pp. 1–7. [Google Scholar] [CrossRef] [Scilit]
- Van der Pol, E.; Oliehoek, F.A. Coordinated deep reinforcement learners for traffic light control. In Proceedings of the 30th Conference on Neural Information Processing Systems (NeurIPS 2016), Barcelona, Spain, 10 December 2016; Volume 8, pp. 21–38. [Google Scholar]
- Li, L.; Lv, Y.; Wang, F.Y. Traffic signal timing via deep reinforcement learning. IEEE/CAA J. Autom. Sin. 2016, 3, 247–254. [Google Scholar] [CrossRef] [Scilit]
- Prashanth, L.A.; Bhatnagar, S. Reinforcement learning with average cost for adaptive control of traffic lights at intersections. In Proceedings of the 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC), Washington, DC, USA, 5–7 October 2011; pp. 1640–1645. [Google Scholar] [CrossRef] [Scilit]
- Chen, X.; Xiong, G.; Lv, Y.; Chen, Y.; Song, B.; Wang, F.Y. A Collaborative Communication-Qmix Approach for Large-scale Networked Traffic Signal Control. In Proceedings of the 2021 IEEE International Intelligent Transportation Systems Conference (ITSC), Indianapolis, IN, USA, 19–22 September 2021; pp. 3450–3455. [Google Scholar] [CrossRef] [Scilit]
- Wei, H.; Zheng, G.; Yao, H.; Li, Z. IntelliLight: A Reinforcement Learning Approach for Intelligent Traffic Light Control. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Minin; Association for Computing Machinery: New York, NY, USA, 2018; pp. 2496–2505. [Google Scholar] [CrossRef] [Scilit]
- Wei, H.; Xu, N.; Zhang, H.; Zheng, G.; Zang, X.; Chen, C.; Zhang, W.; Zhu, Y.; Xu, K.; Li, Z. CoLight: Learning Network-level Cooperation for Traffic Signal Control. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, Beijing, China, 3–7 November 2019; pp. 1913–1922. [Google Scholar] [CrossRef] [Scilit]
- Zheng, G.; Xiong, Y.; Zang, X.; Feng, J.; Wei, H.; Zhang, H.; Li, Y.; Xu, K.; Li, Z. Learning Phase Competition for Traffic Signal Control. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, Beijing, China, 3–7 November 2019; pp. 1963–1972. [Google Scholar] [CrossRef] [Scilit]
- Mei, H.; Li, J.; Shi, B.; Wei, H. Reinforcement learning approaches for traffic signal control under missing data. arXiv 2023, arXiv:2304.10722. [Google Scholar]
- Xu, D.; Li, C.; Wang, D.; Gao, G. Robustness Analysis of Discrete State-Based Reinforcement Learning Models in Traffic Signal Control. IEEE Trans. Intell. Transp. Syst. 2023, 24, 1727–1738. [Google Scholar] [CrossRef] [Scilit]
- Haydari, A.; Zhang, M.; Chuah, C.N. Adversarial Attacks and Defense in Deep Reinforcement Learning (DRL)-Based Traffic Signal Controllers. IEEE Open J. Intell. Transp. Syst. 2021, 2, 402–416. [Google Scholar] [CrossRef] [Scilit]
- Sun, Z.; Jia, X.; Cai, Y.; Ji, A.; Lin, X.; Liu, L.; Wang, W.; Tu, Y. Joint control of traffic signal phase sequence and timing: A deep reinforcement learning method. Digit. Transp. Saf. 2025, 4, 118–126. [Google Scholar] [CrossRef] [Scilit]
- Arafat, M.; Sadeghvaziri, E.; Javid, R. Performance evaluation of red-light violation warning application with traffic signal actuation and coordination. Digit. Transp. Saf. 2025, 4, 108–117. [Google Scholar] [CrossRef] [Scilit]
- Mei, H.; Lei, X.; Da, L.; Shi, B.; Wei, H. Libsignal: An open library for traffic signal control. Mach. Learn. 2024, 113, 5235–5271. [Google Scholar]
- Lopez, P.A.; Behrisch, M.; Bieker-Walz, L.; Erdmann, J.; Flötteröd, Y.P.; Hilbrich, R.; Lücken, L.; Rummel, J.; Wagner, P.; Wiessner, E. Microscopic Traffic Simulation using SUMO. In Proceedings of the 2018 21st International Conference on Intelligent Transportation Systems (ITSC), Maui, HI, USA, 4–7 November 2018; pp. 2575–2582. [Google Scholar] [CrossRef] [Scilit]
- Wei, H.; Zheng, G.; Gayah, V.; Li, Z. A survey on traffic signal control methods. arXiv 2019, arXiv:1904.08117. [Google Scholar]
- Ault, J.; Sharon, G. Reinforcement Learning Benchmarks for Traffic Signal Control. In Proceedings of the Thirty-Fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1), Virtual, 1 May–8 June 2021. [Google Scholar]
- Wei, H.; Chen, C.; Zheng, G.; Wu, K.; Gayah, V.; Xu, K.; Li, Z. PressLight: Learning Max Pressure Control to Coordinate Traffic Signals in Arterial Network. In Proceedings of the 25th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Anchorage, AK, USA, 4–8 August 2019; pp. 1290–1298. [Google Scholar] [CrossRef] [Scilit]
- Zheng, G.; Zang, X.; Xu, N.; Wei, H.; Yu, Z.; Gayah, V.; Xu, K.; Li, Z. Diagnosing reinforcement learning for traffic signal control. arXiv 2019, arXiv:1905.04716. [Google Scholar]
- Chu, T.; Wang, J.; Codecà, L.; Li, Z. Multi-Agent Deep Reinforcement Learning for Large-Scale Traffic Signal Control. IEEE Trans. Intell. Transp. Syst. 2020, 21, 1086–1095. [Google Scholar] [CrossRef] [Scilit]
- Chen, C.; Wei, H.; Xu, N.; Zheng, G.; Yang, M.; Xiong, Y.; Xu, K.; Li, Z. Toward a thousand lights: Decentralized deep reinforcement learning for large-scale traffic signal control. In Proceedings of the AAAI conference on artificial intelligence, New York, NY, USA, 7–8 February 2020; Volume 34, pp. 3414–3421. [Google Scholar]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.










