Search Results (17)

To address the needs of enhancing adaptive control and reducing emissions at intersections within intelligent traffic signal systems, this study innovatively proposes a deep reinforcement learning signal control model tailored for mixed traffic flows. Addressing shortcomings in existing models that overlook mixed traffic scenarios, neglect optimization of CO₂ emissions, and overly rely on high-performance algorithms, our model utilizes vehicle queue length, average speed, numbers of gasoline and electric vehicles, and signal phases as state information. It employs a fixed-phase strategy to decide between maintaining or switching signal states and incorporates a reward function that balances vehicle CO₂ emissions and waiting times, significantly lowering intersection carbon emissions. Following training with reinforcement learning algorithms, the model consistently demonstrates effective control outcomes. Simulation results using the SUMO platform reveal that our designed reward mechanism facilitates the rapid and stable convergence of intelligent agents. Compared with Fixed Time Control (FTC), Actuated Traffic Signal Control (ATSC), and Fuel-ECO TSC (FECO-TSC) methods, our model achieves superior performance in average waiting times and CO₂ emissions. Even across scenarios with gasoline–electric vehicle ratios of 25–75%, 50–50%, and 75–25%, the model exhibits significant advantages. These simulations validate the model’s rationality and effectiveness in promoting low-carbon travel and efficient signal control. Full article

Multi-Agent Reinforcement Learning (MARL) presents a promising approach for addressing the complexity of Traffic Signal Control (TSC) in urban environments. However, existing platforms for MARL-based TSC research face challenges such as slow simulation speeds and convoluted, difficult-to-maintain codebases. To address these limitations, we introduce PyTSC, a robust and flexible simulation environment that facilitates the training and evaluation of MARL algorithms for TSC. PyTSC integrates multiple simulators, such as SUMO and CityFlow, and offers a streamlined API, enabling researchers to explore a broad spectrum of MARL approaches efficiently. PyTSC accelerates experimentation and provides new opportunities for advancing intelligent traffic management systems in real-world applications. Full article

Reducing traffic energy consumption is crucial for smart cities, and vehicle carbon emissions are a key energy indicator. Traffic signal control (TSC) is a useful method because it can affect the energy consumption of vehicles on the road by controlling the stop-and-go of vehicles at traffic intersections. However, setting traffic signals to reduce energy consumption will affect traffic efficiency and this is not in line with traffic management objectives. Current studies adopt multi-objective optimization methods with high traffic efficiency and low carbon emissions to solve this problem. However, most methods use static weights, which cannot adapt to complex and dynamic traffic states, resulting in non-optimal performance. Current energy indicators for urban transportation often fail to consider passenger fairness. This fairness is significant because the purpose of urban transportation is to serve people’s mobility needs not vehicles. Therefore, this paper proposes Multi-objective Adaptive Meta-DQN TSC (MMD-TSC), which introduces a dynamic weight adaptation mechanism to simultaneously optimize traffic efficiency and energy saving, and incorporates the per capita carbon emissions as the energy indicator. Firstly, this paper integrates traffic state data such as vehicle positions, velocities, vehicle types, and the number of passengers and incorporates fairness into the energy indicators, using per capita carbon emissions as the target for reducing energy consumption. Then, it proposes MMD-TSC with dynamic weights between energy consumption and traffic efficiency as reward functions. The MMD-TSC model includes two agents, the TSC agent and the weight agent, which are responsible for traffic signal adjustment and weight calculation, respectively. The weights are calculated by a function of traffic states. Finally, the paper describes the design of the MMD-TSC model learning algorithm and uses a SUMO (Simulation of Urban Mobility) v.1.20.0 for traffic simulation. The results show that in non-highly congested traffic states, the MMD-TSC model has higher traffic efficiency and lower energy consumption compared to static multi-objective TSC models and single-objective TSC models, and can adaptively achieve traffic management objectives. Compared with using vehicle average carbon emissions as the energy consumption indicator, using per capita carbon emissions achieves Pareto improvements in traffic efficiency and energy consumption indicators. The energy utilization efficiency of the MMD-TSC model is improved by 35% compared to the fixed-time TSC. Full article

Public transportation has been identified as a viable solution to mitigate traffic congestion. Transit signal priority (TSP) control, which is widely used at signalized intersections, has been recognized as a practical strategy to improve the efficiency and reliability of bus operations. However, traditional TSP control may fall short of efficiency and is facing several challenges of negative externalities for non-transit users and the need to handle conflicting priority requests. Recent studies have proposed the use of reinforcement learning (RL) methods to identify efficient traffic signal control (TSC). Some of these studies on RL-based TSC have incorporated the concept of max-pressure (MP), which is a maximal weight-matching algorithm to minimize queue sizes. Nevertheless, the existing RL-based TSC methods focus on private vehicles and cannot adequately distinguish between buses and private vehicles. In prior research, RL-based control has been implemented within the context of bus rapid transit (BRT) systems. This study proposes a novel RL-based TSC strategy that leverages the MP concept and extends it to incorporate TSP control. This is the first implementation of RL-based TSP control within the mixed-traffic road network. A significant innovation of this research is the introduction of the priority factor (PF), which is designed to prioritize bus movements at signalized intersections. The proposed RL-based TSP with PF control seeks to balance the competing objectives of enhancing bus operations while mitigating adverse impacts on non-transit users. To evaluate the performance of the proposed TSP method with the PF mechanism, simulations were conducted on an arterial and a grid network under dynamic traffic conditions. The simulation results demonstrated that the proposed TSP with PF not only reduces bus travel times and resolves conflicts between priority requests but also does not make a significant negative impact on passenger car operations. Furthermore, the PF can be dynamically assigned according to the number of passengers on each bus, suggesting the potential for the proposed approach to be applied in various traffic management scenarios. Full article

The exponential increase in vehicles, quick urbanization, and rising demand for transportation are straining the world’s road infrastructure today. To have a sustainable transportation system with dynamic traffic volume, an Adaptive Traffic Signal Control system (ATSC) should be contemplated to reduce urban traffic congestion and, thus, help reduce the carbon footprints/emissions of greenhouse gases. With dynamic cleave, the ATSC system can adapt the signal timing settings in real-time according to seasonal and short-term variations in traffic demand, enhancing the effectiveness of traffic operations on urban road networks. This paper provides a comprehensive study on the insights, technical lineaments, and status of various research work in ATSC. In this paper, the ATSC is categorized based on several road intersections (RIs), viz., single-intersection (SI) and multiple-intersection (MI) techniques, viz., Fuzzy Logic (FL), Metaheuristic (MH), Dynamic Programming (DP), Reinforcement Learning (RL), Deep Reinforcement Learning (DRL), and hybrids used for developing Traffic Signal Control (TSC) systems. The findings from this review demonstrate that modern ATSC systems designed using various techniques offer substantial improvements in managing the dynamic density of the traffic flow. There is still a lot of scope to research by increasing the number of RIs while designing the ATSC system to suit real-life applications. Full article

Multi-Agent Reinforcement Learning (MARL) has shown strong advantages in urban multi-intersection traffic signal control, but it also suffers from the problems of non-smooth environment and inter-agent coordination. However, most of the existing research on MARL traffic signal control has focused on designing efficient communication to solve the environment non-smoothness problem, while neglecting the coordination between agents. In order to coordinate among agents, this paper combines MARL and the regional mixed-strategy Nash equilibrium to construct a Deep Convolutional Nash Policy Gradient Traffic Signal Control (DCNPG-TSC) model, which enables agents to perceive the traffic environment in a wider range and achieves effective agent communication and collaboration. Additionally, a Multi-Agent Distributional Nash Policy Gradient (MADNPG) algorithm is proposed in this model, which is the first time the mixed-strategy Nash equilibrium is used for the improvement in the Multi-Agent Deep Deterministic Policy Gradient algorithm traffic signal control strategy to provide the optimal signal phase for each intersection. In addition, the eco-mobility concept is integrated into MARL traffic signal control to reduce pollutant emissions at intersections. Finally, simulation results in synthetic and real-world traffic road networks show that DCNPG-TSC outperforms other state-of-the-art MARL traffic signal control methods in almost all performance metrics, because it can aggregate the information of neighboring agents and optimize the agent’s decisions through gaming to find an optimal joint equilibrium strategy for the traffic road network. Full article

This paper presents our research on a traffic signal control system (TSCS) at V2X intersections. The overall objective of the study is to create an implementable TSCS. The specific objective of this paper is to investigate a distributed system towards implementation. The objective function of minimizing queue delay is formulated as the integral of queue lengths. The discrete queueing estimation is mixed with macro and micro traffic flow models. The novel proposed architecture alleviates the communication network bandwidth constraint by processing BSMs and computing queue lengths at the local intersection. In addition, a two-stage distributed system is designed to optimize offsets, splits, and cycle length simultaneously and in real time. The paper advances TSCS theories by contributing a novel analytic formulation of delay functions and their first degree of derivatives for a two-stage optimization model. The open-source traffic simulation engine Enhanced Transportation Flow Open-Source Microscopic Model (ETFOMM version 1.2) was selected as a simulation environment to develop, debug, and evaluate the models and the system. The control delay of the major direction, minor direction, and the total network were collected to assess the system performance. Compared with the optimized TSCS timing plan by the Virginia Department of Transportation, the system generated a 21% control delay reduction in the major direction and a 7% control delay reduction in the minor direction at just a 10% penetration rate of connected vehicles. Finally, the proposed distributed and centralized systems present similar performances in the case study. Full article

Effective traffic signal control (TSC) plays an important role in reducing vehicle emissions and improving the sustainability of the transportation system. Recently, the feasibility of using multi-agent reinforcement learning technology for TSC has been widely verified. However, the process of mapping road network states onto actions has encountered many challenges, due to the limited communication between agents and the partial observability of the traffic environment. To address this problem, this paper proposes a communication-enhancement value decomposition, multi-agent reinforcement learning TSC method (CVDMARL). The model combines two communication methods: implicit and explicit communication, decouples the complex relationships among the multi-signal agents through the centralized-training and decentralized-execution paradigm, and uses a modified deep network to realize the mining and selective transmission of traffic flow features. We compare and analyze CVDMARL with six different baseline methods based on real datasets. The results show that compared to the optimal method MN_Light, among the baseline methods, CVDMARL’s queue length during peak hours was reduced by 9.12%, the waiting time was reduced by 7.67%, and the convergence algebra was reduced by 7.97%. While enriching the information content, it also reduces communication overhead and has better control effects, providing a new idea for solving the collaborative control problem of multi-signalized intersections. Full article

Mixed traffic flows are opening up new areas for research and are seen as key drivers in the field of data and services that will make roads safer and more environmentally friendly. Understanding the effects of Connected Vehicles (CVs) and Connected Autonomous Vehicles (CAVs), as one of the vehicle components of mixed traffic flows, will make it easier to avoid traffic congestion and contribute to the creation of innovative applications and solutions. It is notable that the literature related to the analysis of the impact of mixed traffic flows on traffic signal control in urban areas rarely considers mixed traffic flow containing CVs, CAVs, and Human Driven Vehicles (HDVs). Therefore, this paper provides an overview of the relevant research papers covering the topic of urban Traffic Signal Control (TSC) and mixed traffic flows. Best practices for intersection state estimation and TSC in the case of mixed traffic flows in an urban environment are summarized and possible approaches for utilizing CVs and CAVs as mobile sensors and actuators are discussed. Full article

Reinforcement-learning (RL) algorithms have made great achievements in many scenarios. However, in large-scale traffic signal control (TSC) scenarios, RL still falls into local optima when controlling multiple signal lights. To solve this problem, we propose a novel goal-based multi-agent hierarchical model (GMHM). Specifically, we divide the traffic environment into several regions. The region contains a virtual manager and several workers who control the traffic lights. The manager assigns goals to each worker by observing the environment, and the worker makes decisions according to the environment state and the goal. For the worker, we adapted the goal-based multi-agent deep deterministic policy gradient (MADDPG) algorithm combined with hierarchical reinforcement learning. In this way, we simplify tasks and allow agents to cooperate more efficiently. We carried out experiments on both grid traffic scenarios and real-world scenarios in the SUMO simulator. The experimental results show the performance advantages of our algorithm compared with state-of-the-art algorithms. Full article

Optimizing traffic signal control (TSC) at intersections continues to pose a challenging problem, particularly for large-scale traffic networks. It has been shown in past research that it is feasible to optimize the operations of individual TSC systems or a small collection of such systems. However, it has been computationally difficult to scale these solution approaches to large networks partly due to the curse of dimensionality that is encountered as the number of intersections increases. Fortunately, recent studies have recognized the potential of exploiting advancements in deep and reinforcement learning to address this problem, and some preliminary successes have been achieved in this regard. However, facilitating such intelligent solution approaches may require large amounts of infrastructure investments such as roadside units (RSUs) and drones, to ensure that connectivity is available across all intersections in the large network. This represents an investment that may be burdensome for the road agency. As such, this study builds on recent work to present a scalable TSC model that may reduce the number of enabling infrastructure that is required. This is achieved using graph attention networks (GATs) to serve as the neural network for deep reinforcement learning. GAT helps to maintain the graph topology of the traffic network while disregarding any irrelevant information. A case study is carried out to demonstrate the effectiveness of the proposed model, and the results show much promise. The overall research outcome suggests that by decomposing large networks using fog nodes, the proposed fog-based graphic RL (FG-RL) model can be easily applied to scale into larger traffic networks. Full article

The effective control and management of traffic at intersections is a challenging issue in the transportation system. Various traffic signal management systems have been developed to improve the real-time traffic flow at junctions, but none of them have resulted in a smooth and continuous traffic flow for dealing with congestion at road intersections. Notwithstanding, the procedure of synchronizing traffic signals at nearby intersections is complicated due to numerous borders. In traditional systems, the direction of movement of vehicles, the variation in automobile traffic over time, accidents, the passing of emergency vehicles, and pedestrian crossings are not considered. Therefore, synchronizing the signals over the specific route cannot be addressed. This article explores the key role of real-time traffic signal control (TSC) technology in managing congestion at road junctions within smart cities. In addition, this article provides an insightful discussion on several traffic light synchronization research papers to highlight the practicability of networking of traffic signals of an area. It examines the benefits of synchronizing the traffic signals on various busy routes for the smooth flow of traffic at intersections. Full article

The traffic signal control problem is an extensively researched area providing different approaches, from classic methods to machine learning based ones. Different aspects can be considered to find an optima, from which this paper emphasises emission reduction. The core of our solution is a novel rewarding concept for deep reinforcement learning (DRL) which does not utilize any reward shaping, hence exposes new insights into the traffic signal control (TSC) problem. Despite the omission of the standard measures in the rewarding scheme, the proposed approach can outperform a modern actuated control method in classic performance measures such as waiting time and queue length. Moreover, the sustainability of the realized controls is also placed under investigation to evaluate their environmental impacts. Our results show that the proposed solution goes beyond the actuated control not just in the classic measures but in emission-related measures too. Full article

Traffic congestion has several causes, including insufficient road capacity, unrestricted demand and improper scheduling of traffic signal phases. A great variety of efforts have been made to properly program such phases. Some of them are based on traditional transportation assumptions, and others are adaptive, allowing the system to learn the control law (signal program) from data obtained from different sources. Reinforcement Learning (RL) is a technique commonly used in previous research. However, properly determining the states and the reward is key to obtain good results and to have a real chance to implement it. This paper proposes and implements a traffic signal control system (TSCS), detailing its development stages: (a) Intelligent Transportation System (ITS) architecture design for the TSCS; (b) design and development of a system prototype, including an RL algorithm to minimize the vehicle queue at intersections, and detection and calculation of such queues by adapting a computer vision algorithm; and (c) design and development of system tests to validate operation of the algorithms and the system prototype. Results include the development of the tests for each module (vehicle queue measurement and RL algorithm) and real-time integration tests. Finally, the article presents a system simulation in the context of a medium-sized city in a developing country, showing that the proposed system allowed reduction of vehicle queues by 29%, of waiting time by 50%, and of lost time by 50%, when compared to fixed phase times in traffic signals. Full article

Recent research works on intelligent traffic signal control (TSC) have been mainly focused on leveraging deep reinforcement learning (DRL) due to its proven capability and performance. DRL-based traffic signal control frameworks belong to either discrete or continuous controls. In discrete control, the DRL agent selects the appropriate traffic light phase from a finite set of phases. Whereas in continuous control approach, the agent decides the appropriate duration for each signal phase within a predetermined sequence of phases. Among the existing works, there are no prior approaches that propose a flexible framework combining both discrete and continuous DRL approaches in controlling traffic signal. Thus, our ultimate objective in this paper is to propose an approach capable of deciding simultaneously the proper phase and its associated duration. Our contribution resides in adapting a hybrid Deep Reinforcement Learning that considers at the same time discrete and continuous decisions. Precisely, we customize a Parameterized Deep Q-Networks (P-DQN) architecture that permits a hierarchical decision-making process that primarily decides the traffic light next phases and secondly specifies its the associated timing. The evaluation results of our approach using Simulation of Urban MObility (SUMO) shows its out-performance over the benchmarks. The proposed framework is able to reduce the average queue length of vehicles and the average travel time by 22.20% and 5.78%, respectively, over the alternative DRL-based TSC systems. Full article