Optimizing Autonomous Vehicle Navigation Through Reinforcement Learning in Dynamic Urban Environments

Abstract

Autonomous vehicle (AV) navigation in dynamic urban environments faces challenges such as unpredictable traffic conditions, varying road user behaviors, and complex road networks. This study proposes a novel reinforcement learning-based framework that enhances AV decision making through spatial-temporal context awareness. The framework integrates Proximal Policy Optimization (PPO) and Graph Neural Networks (GNNs) to effectively model urban features like intersections, traffic density, and pedestrian zones. A key innovation is the urban context-aware reward mechanism (UCARM), which dynamically adapts the reward structure based on traffic rules, congestion levels, and safety considerations. Additionally, the framework incorporates a Dynamic Risk Assessment Module (DRAM), which uses Bayesian inference combined with Markov Decision Processes (MDPs) to proactively evaluate collision risks and guide safer navigation. The framework’s performance was validated across three datasets—Argoverse, nuScenes, and CARLA. Results demonstrate significant improvements: An average travel time of 420 ± 20 s, a collision rate of 3.1%, and energy consumption of 11,833 ± 550 J in Argoverse; 410 ± 20 s, 2.5%, and 11,933 ± 450 J in nuScenes; and 450 ± 25 s, 3.6%, and 13,000 ± 600 J in CARLA. The proposed method achieved an average navigation success rate of 92.5%, consistently outperforming baseline models in safety, efficiency, and adaptability. These findings indicate the framework’s robustness and practical applicability for scalable AV deployment in real-world urban traffic conditions.

1. Introduction

1.1. Background

Self-driving cars have the potential to reshape urban transportation by reducing traffic congestion, enhancing safety, and improving energy efficiency [1]. However, driving in dynamic urban environments remains a significant research challenge due to the uncertainty of traffic flows [2], road conditions [3], and interactions with surrounding elements through Vehicle-to-Everything (V2X) communication [4]. These factors demand adaptive models that can respond in real time to changing scenarios [5]. Reinforcement learning (RL) has emerged as a powerful tool to address these complexities, enabling autonomous vehicles to learn optimal strategies through exploration in uncertain environments [6]. RL has shown promise in areas such as V2X coordination and traffic flow optimization, where vehicles must make collaborative decisions in dense, interactive settings [7,8]. Despite the projected growth of AV adoption at 40% annually through the 2030s [9], safety concerns remain a key barrier. AVs are projected to grow at a compound annual growth rate of 40% this decade (2030s) (see Figure 1) [10]. Despite this growth, safety concerns remain a significant barrier.

A report by the National Highway Traffic Safety Administration states that over 90% of road accidents in the U.S. are caused by human error. Autonomous vehicles (AVs) are expected to mitigate these incidents and can reduce fuel consumption by up to 20% through optimized routing and integrated traffic flow management [11].

1.2. Research Gap and Challenges

Autonomous vehicle navigation in urban environments is challenged by unpredictable traffic, variable road conditions, and diverse road users [1]. Existing systems often fail to adapt in real time, leading to inefficient routing, increased energy use, and safety concerns [12]. While reinforcement learning (RL) offers potential, many models lack spatial-temporal awareness and overlook critical urban factors like traffic density and pedestrian behavior [13]. To address these gaps, this research proposes the Urban Reinforcement Learning Navigation Framework (URLNF), a hybrid model combining PPO and GNNs. URLNF integrates two key modules—UCARM and DRAM—for real-time adaptation and decision optimization. This framework enhances scalability, safety, and energy efficiency in complex urban scenarios by embedding context-aware rewards and proactive risk assessment. Figure 2 shows a typical flow architectural diagram of a traditional RL-controlled AV-based urban traffic management system.

1.3. Proposed Solution and Objectives

This study aims to optimize autonomous vehicle navigation in complex urban environments by applying the Urban Reinforcement Learning Navigation Framework (URLNF). The framework minimizes travel time, improves energy efficiency, and ensures safety by learning optimal policies under uncertainty. Our approach addresses real-world constraints such as dynamic traffic, signal-based routing, energy usage, and adaptive lane priorities.

The main goal of this paper is to develop a novel reinforcement learning-based framework for optimizing autonomous vehicle navigation in dynamic urban environments, enhancing safety and efficiency in real-world traffic conditions. The objectives of this study are:

(a)

To develop a hybrid DRL model (i.e., URLNF) integrating PPO with GNN-based spatial temporal representation to effectively capture and utilize the complex interactions in dynamic urban traffic scenarios.

(b)

To formulate a reward system that dynamically adapts traffic conditions, safety constraints, and environmental factors to guide AVs towards safer and more efficient navigation.

(c)

To incorporate Bayesian techniques to predict potential collision risk and enable proactive decision making for mitigating hazards in highly dynamic settings.

(d)

To implement and evaluate the framework using advanced simulation tools and real-world traffic datasets, focusing on metrices like travel time, collision rate, and energy efficiency.

This paper is structured in five sections: The introduction outlines the challenges of AV navigation in urban environments, highlights the research gap, and states the objectives of the study. The literature review explores existing methods and technologies in RL-based navigations, identifying their limitations. The methodology details the design and implementations of a novel hybrid RL-based framework including UCARM and DRAM. The results and discussion present the evaluation through simulations and dataset validations analyzing the performance improvements in efficiency, safety, and energy metrics. The conclusions summarize the findings, discuss the implications, and suggest potential future directions.

2. Literature Review

2.1. Autonomous Vehicles (AVs) in Urban Traffic Systems

The placement of autonomous vehicles in urban traffic systems has emerged as a critical area of research with reinforcement learning techniques taking a central role in addressing challenges related to navigation, coordination, and traffic optimization [14]. In one study [14], the author introduced a multi-objective reinforcement learning framework designed for autonomous drone navigation in urban environments with wind zones, a methodology readily applicable to AVs. Their multi-objective reinforcement learning (MORL) model demonstrated the ability to balance key objectives such as safety, energy improvement, and travel time optimization. However, the author mentioned the computational intensity of MORL, especially for real-time application, which presents an important limitation for scaling this approach to a dense urban traffic system. In another study [15], the author imposed multi-agent reinforcement to increase the efficiency of AV fleets in smart cities. Their model used cooperative behavior among AVs, leading to improved traffic flow and reduced congestion. The author admitted to scalability challenges when trying to visualize these behaviors, especially as the numbers of agents increased in complex urban networks, where coordination and communication become increasingly difficult.

In another work [16], the author found a comprehensive review of deep reinforcement learning for path planning in AVs, focusing on effectiveness in dynamic urban environments characterized by unpredictable traffic conditions and obstacles changing road layouts. The DR-based algorithms were shown to optimize vehicle routes in real time, ensuring both safety and efficiency. The author also observed that the DRL system faced significant drawbacks due to its reliance on a large dataset and high computational resources, complicating its practical deployment in a real-world system. In a similar study [17], the author described adaptive speed planning for unmanned vehicles using DRL, enabling vehicles to adjust their speed dynamically based on real-time traffic conditions and safety considerations. Thus, this approach mainly improved traffic flow efficiency and identified limitations in handling extreme unknown scenarios like sudden pedestrian crossings or accidents, which are frequent in urban areas.

The literature [18] shows that the integration of Distributed Reinforcement Learning (DRL) with Model Predictive Control (MPC) can be used to manage autonomous vehicle groups in urban roads networks. This approach accessed synchronized vehicle movement, enhancing traffic data and reducing fuel consumption. Thus, the system’s reliance on robust vehicle-to-infrastructure communications increased risk, as communication failure could lead to significant disruptions and reduced safety. To address another challenge, in [19], the author made an uncertainty-aware DRL model for crowd navigation in a shared urban space. This framework allowed AVs to navigate effectively in environments with dynamics hurdles like pedestrians. While the model performed well in controlled settings, its efficiency decreased in scenarios with heightened environment uncertainty, such as dense pedestrian zones or chaotic traffic.

Current research on autonomous vehicle (AV) navigation in urban environments focuses on the integration of ML and reinforcement to address the challenges such as multi-objective optimization, real-time decision making, and system robustness. In [20], the author improved traffic flow using RL for connected and automated vehicles (CAVs) at intersections and found that reliable coordination in dense traffic is a limitation. The author of [21] introduced a multi-objective RL framework optimizing safety and improvements for drones, but scalability to complex urban networks remained a challenge in autonomous drones. In another study [22], ML was applied for 3D routing optimization in UAVs, with the potential for AVs, but multi-agent coordination in urban settings requires further development. Deep reinforcement learning approaches like for UAV hurdle avoidance and using decision transformers for behavior prediction have shown promise but face real-time computational limits and unpredictability challenges. Similarly, another study [23] combined improved DRL with autonomous UAV visual navigation using traditional control methods for adaptability, but scalability issues persisted. Another author described and observed a robust RL framework for AV safety under uncertain conditions, but balancing robustness with efficiency in noisy urban environments remained unresolved [24].

Table 1. Summary of autonomous vehicle studies in urban traffic systems.

Study	Dataset	Approach	Limitation
[7]	Custom simulation	Deep RL-based adaptive speed planning	Focuses only on speed, not full decision pipeline
[16]	Not specified	AI-driven integration for smart city AVs	Generalized concept, lacks implementation detail
[18]	Urban road network models	Joint distributed RL + MPC for platoons	Limited scalability for large platoons
[19]	Shared space simulation	Uncertainty-aware DRL crowd navigation	No real-world validation yet
[20]	Signalized intersection sim	Multi-objective RL control for CAVs	Limited generalization across cities
[21]	Urban drone environments	Autoencoder fusion for drone aerodynamics	Applies to aerial rather than ground vehicles

shows the summary of autonomous vehicle studies in urban traffic systems.

2.2. Reinforcements Learning for Traffic Optimization

Simultaneously, progress in decision making for advanced autonomous vehicles has been propelled by intelligent systems that can reason based on driver behavior and environmental context. The author of [25] investigated a decision-making model for autonomous cars driven by driver intelligence, integrating environmental reasoning for enhanced adaptability and realism in navigation. Their investigation revealed that integrating contextual intelligence enables autonomous systems to improve long-term decision making in urban traffic, hence augmenting safety and operational efficiency [26]. Nonetheless, the computational expense of handling substantial amounts of contextual data continues to be a constraint in real-time applications. In [27], the author presented deep attention-driven RL (DAD-RL), an innovative framework that use attention mechanisms to concentrate on the most pertinent environmental aspects during decision making. This method demonstrated robust efficiency in dynamic, intricate contexts by enhancing decision-making precision and minimizing processing demands [28]. Nonetheless, the dependence on attention mechanisms complicates model training, especially in edge-case situation that necessitate intricate decision making in uncommon contexts.

In addition to RL-based approaches, recent research has explored fuzzy optimization and metaheuristic algorithms to address navigation and decision-making challenges in dynamic urban systems. The study by Sutikno [29] presents a comparative analysis of fuzzy logic and metaheuristic strategies for energy-aware route planning, demonstrating their robustness and adaptability in uncertain environments. The integration of such techniques with RL could offer hybrid models capable of better handling multi-objective constraints such as safety, fuel economy, and travel time in real-time navigation scenarios [30,31].

Table 2. Summary of reinforcement learning techniques for traffic optimization.

Study	Dataset	Approach	Limitation
[25]	Simulated AV data with perturbations	Robust RL against worst-case observation noise	Focuses only on observational noise, not actuator issues
[26]	Autonomous driving navigation datasets	Decision transformer for RL-based navigation	Limited to learning from offline trajectories
[27]	Custom dynamic driving environment (simulated)	Deep attention-driven RL for dynamic decision making	Lacks real-world or physical environment deployment
[29]	Not dataset-specific; optimization theory-based	Fuzzy logic and metaheuristic optimization algorithms	Not tailored for AVs; focuses on general optimization

shows the summary of reinforcement learning techniques for traffic optimization.

2.3. Urban Context-Aware Decision Making for Autonomous Vehicles

Recent improvements in urban context-aware decision making for autonomous vehicles have demonstrated significant potential in enhancing traffic flow and vehicle coordination, especially in intricate metropolitan settings with heterogeneous traffic. In [28], the author introduced an adaptive signal control connected and automated vehicle (CAV) coordination system employing deep reinforcement learning (DRL) to regulate mixed traffic at signalized intersections. This system dynamically modifies signal timing and synchronizes connected autonomous vehicles (CAVs) to reduce congestion and enhance traffic flow. The findings indicated substantial decreases in delays and enhancements in intersection efficiency; nevertheless, the model’s reliance on precise vehicle connectivity and real-time traffic data poses obstacle for implementation in regions with unreliable communication infrastructure. Likewise, [32] concentrated on reinforcement learning for traffic signal control, utilizing a state reduction method to streamline the state space and enhance the system’s computing efficiency. This method demonstrated superior performance compared to conventional signal controllers; nevertheless, its effectiveness in high-density traffic scenarios during peak hours was constrained by difficulties in scaling the system to accommodate bigger, more intricate networks.

Subsequent research has combined graph-based methodologies with multi-agent reinforcement learning (MARL) to enhance decision making in urban traffic situations. Xing et al. [33] presented GRL-GCN, a hybrid model that integrates reinforcement learning with graph convolutional networks (GCNs) for traffic flow production in smart cities. This method exhibited significant accuracy in simulating traffic dynamics; nevertheless, its efficiency may diminish in highly dynamic environments characterized by fast fluctuations in traffic patterns. Conversely, Ref. [34] formulated a distributed control framework utilizing reinforcement learning for collaborative intersection management, wherein each agent (vehicle or traffic signal) learns autonomously while cooperating with others to enhance the overall system efficiency. Their methodology enhances traffic efficiency; yet, the difficulty persists in guaranteeing that the distributing agents can manage conflicts and sustain when integrating with numerous automobiles or traffic signals.

The researcher in [35] conducted a survey on multi-agent reinforcement learning (MARL) for connected and automated cars, emphasizing the significance of communication between vehicle and infrastructure for optimal decision making. Notwithstanding the positive outcomes, the incorporation of numerous agents in extensive urban networks persists in encountering difficulties related to coordination, communication overhead, and the maintenance of stable learning among agents [36]. Research indicates that although reinforcement learning and multi-agent systems are effective for enhancing urban traffic management, their particle implementation is impeded by challenges concerning scalability, real-time data processing, an inter-agent communication in intricate environments.

Recent advancements further emphasize multi-objective optimization and robust perception in autonomous driving. For instance, the “Multi-Objective Autonomous Eco-Driving Strategy” proposes balancing efficiency and environmental impact, while recent reviews on occluded object detection highlight perception challenges in dense urban settings. A survey on RL-based highway AV control outlines techniques for stable high-speed decision making, complementing the urban focus of URLNF.

Table 3. Comparative table of previous studies.

Studies	Technique	Results	Limitations	Findings
[14]	MORL	Optimized safety, energy efficiency, and travel time in urban drone navigation	High computational intensity limits real-time application in dense urban systems	Potential for AV application but needs scalability improvements
[16]	DRL for path plaining	Optimized real-time vehicle routes ensuring safety and efficiency	Dependency on large datasets and computational resources	DRL can enhance path plaining but requires better resource optimization
[17]	Adaptive speed plaining using DRL	Enhanced traffic flow by dynamically adjusting vehicle speeds	Ineffective in extreme situations like sudden pedestrian crossings or accidents	Effective for dynamic speed adjustments; need improvements for rare situations
[18]	Distributed DRL with MPC	Synchronized vehicle movements, enhancement of data, and reduced fuel consumption	Reliance on robust vehicle-to-infrastructure communication creates risk during failures	Integration of MPC with DRL shows promise but is vulnerable to communication issues
[19]	Uncertainty aware DRL model	Effective navigation in environments with dynamic hurdles	Decrease efficiency in environments with heightened uncertainty	Suitable for controlled settings
[35]	MARL	Improved traffic flow and reduced congestion through cooperative AV behavior	Scalability challenges with increased agents in complex urban networks	Mentioned the need for robust coordination and communication mechanisms

shows the comparative table of previous studies.

3. Methodology

The approach proposed in this investigation is intended to tackle the multifaceted issues associated with AV navigation in dynamic city settings by means of a new hybrid reinforcement learning approach. The framework proposes the integration of PPO with GNN to support efficient decision making and dynamic navigation. The urban environment is represented by a directed graph, where nodes are intersections, vehicles, and pedestrian areas; edges are road segments with specific characteristics, including traffic density, speed limit, and priority factors. GNNs are used for modeling the spatial-temporal dependencies in traffic patterns of the urban environment and facilitate understanding of complex interactions between different features of that environment.

For the purpose of navigation, the PPO algorithm is used, which, while updating the policy, reduces the exploration and increases the exploitation through the clipped surrogate loss function. This allows learning under stiffness constraints to be strong. Also, the framework incorporates a UCARM that adapts the reward system to be dependent on traffic rules, current traffic density data, and safety concerns. This helps to achieve the objective of making the learning process mimic real-life urban navigation objectives such as time to task, energy optimization, and collision prevention.

To this end, a DRAM is incorporated into the framework to assess real-time collision risks with the help of Bayesian inference and Markov Decision Processes (MDPs). This module also drives the risk management by simulating the variability in the movement of pedestrians, traffic, and changes in the physical environment. The entire framework is evaluated on real-world datasets including Argoverse and nuScenes as well as simulated data created in the CARLA simulator. To evaluate the proposed approach, different cases, such as dense traffic, mixed pedestrian areas, and adverse weather conditions, are considered. The findings are used to show how the framework can be used to add safety, decrease energy use, and increase navigation efficiency within cities.

Figure 3 illustrates the core components of the proposed AV navigation framework. It begins with environment sensing and data preprocessing, followed by a GNN-based spatial-temporal encoder. The processed information flows into the PPO-based policy optimizer, enhanced with context-aware reward adjustment and dynamic collision risk assessment. The final output is an optimized decision for real-time vehicle control, enabling safe and efficient navigation across urban scenarios.

3.1. Dataset Description

(a)

Datasets

The experiments in this study utilize three datasets: Argoverse, nuScenes, and CARLA. The Argoverse dataset contains annotated vehicle trajectory data collected from urban roads in Pittsburgh, Pennsylvania, including downtown intersections, multi-lane roads, and signal-controlled junctions. It provides high-definition semantic maps, 3D tracking of surrounding agents, and real-time traffic light data. The nuScenes dataset includes driving data from Boston and Singapore, covering mixed urban traffic, pedestrian zones, and sensor-rich vehicle logs. The CARLA dataset is a synthetic dataset generated using the CARLA simulator, offering controlled conditions across various weather scenarios, road types, and dynamic agent behaviors for evaluating autonomous navigation models.

In order to assess the performance of the proposed URLNF, the research used both high-quality real-world datasets and synthetic ones. Both datasets were selected to provide coverage of a large range of possible urban navigation settings, traffic conditions, road configurations, and interactions with pedestrians. Below is a detailed description of the datasets:

(b)

Argoverse: A recorded driving dataset containing over 320 h of annotated data collected in an urban environment. This dataset provides high-quality 3D tracking of vehicles and their interactions with the road infrastructure: lanes, drivable space, and the surrounding environment; all of which are important for modeling urban dynamics. Argoverse also includes map-based priors to improve decisions made by self-driving automobiles in multi-agent environments.

Figure 4 shows two intersection scenes selected from the Argoverse dataset which indicate that urban traffic is challenging. The left panel shows a motor road intersection with multiple lanes for vehicles, walkways for crossing, automobiles, and a green traffic light. The right panel depicts another intersection scenario where the vehicles, pedestrians, and cyclist are depicted in a setting with a school bus and pedestrians within the vicinity of the crosswalk.

(c)

NuScenes: Another realistic dataset containing full multi-sensor data, LIDAR, radar, and high-resolution camera data. NuScenes provides traffic signal states, vehicle motion, and the overall environment of the scene, and it is useful for model training and testing in complex dynamic traffic environments.

Figure 5 shows the modality of sensor data in the nuScenes dataset, which also includes camera and radar data. The left panel demonstrates six views of a car and its vicinity with bounding boxes for objects detected on them. The right panel shows the radar field of view (FoV), a list of radar detections, and clustered object annotations.

(d)

Synthetic Simulation Data (CARLA): In addition to the real-world datasets, synthetic data was also created from the CARLA simulator. The simulation environment reflects the traffic scenario of a city, weather conditions, and pedestrian actions. This dataset is also good for testing when there is congestion, bad weather, and other conditions that may not be well captured in real-world datasets.

In Figure 6, a simulated urban environment is depicted using the CARLA simulator, where different vehicles, pedestrians, and dynamic weather conditions at a multi-lane intersection are shown. The scene demonstrates the difficulties of orientation in urban environments in a safety bubble.

Table 4. Dataset specifications.

Dataset	Type	Size (GB)	Key Features	Applications
Argoverse	Real-world	400	3D Tracking, Lane Geometry, Map Priors, Road Topology	Multi-Agent Navigation, Urban Road Interactions
nuScenes	Real-world	100	Multi-sensor (Lidar, Radar, Camera), Traffic Signal States	Traffic Behavior Prediction, Sensor Fusion
CARLA	Synthetic Simulation	Variable	Dynamic Traffic Patterns, Pedestrian Behaviors, Weather Variations	Edge-case Scenarios, Reinforcement Learning Testing

shows the Dataset specifications. To prepare the datasets for integration into the framework, the following preprocessing steps were undertaken:

(a)

Normalization and Standardization: All quantitative data, including the vehicle velocities, distances, and the readings from the sensors, were scaled to be compatible between datasets.

(b)

Augmentation: To further increase the data variability, information from additional scenarios such as the weather and time of day were included into CARLA data.

(c)

Data Fusion: In both nuScenes and Argoverse, data from different sensors was combined to create a consolidated view of the environment with LIDAR, radar, and camera data being used for perception.

(d)

Graph Construction: Urban environments were modeled as graphs where nodes correspond to intersections and vehicles while the edges correspond to road segments, traffic density, and signal indications.

The use of both real and synthetic datasets offers multiple benefits for the proposed framework. It allows for modeling of the real traffic phenomena in a urban environment and thus guarantees that the proposed framework can deal with real-world traffic conditions. Moreover, the use of synthetic data enables testing of cases that are not quite frequent in real-world scenarios, for example, rainy or snowy conditions or erratic behavior of pedestrians. This makes the framework scalable and applicable in different layouts of cities and under different climatic conditions that could make it useful in as many navigation problems as possible.

3.2. System Model

The proposed framework models the urban environment as a directed graph $G=(V,E)$, where:

• V stands for the junction points, the car, and places of interest.
• E represents the set of roads between these intersections, where each road is described by features including traffic congestion, signal settings, and speed profile.

Parameter Definitions

Every AV interacts with the shared urban environment through a distinct decision-making process. At any given time $t$, the state of each AV is denoted by $s_t\in S$, while its corresponding action is represented as $u_t\in A$. The evolution of the system is modeled as a generalized Markov Decision Process (MDP) defined by the five elements $(S,A,P,R)$, where:

• $S$ (State Space): Each state $s_t$ captures both the internal status of the AV and its environmental context. This includes the vehicle’s current position, speed, heading angle, and acceleration, along with external observations such as nearby object positions, traffic light status, lane information, and local traffic density. This provides a comprehensive understanding of the AV’s dynamic surroundings.
• $A$ (Action Space): The action $u_t$ refers to the AV’s control commands at time $t$, which consist of throttle (acceleration), brake level, and steering angle. These control signals are generated by the policy model in response to the observed state $s_t$.
• $P$ (Transition Probability): The probabilistic relationship between the current state $s_t$ and the next state $s_{t+1}$ is governed by the selected action $u_t$. This captures the system’s dynamics and how the vehicle’s control choices affect its movement and interactions.
• $R$ (Reward Function): The reward function $R(s_t,a_t)$ quantifies the effectiveness of each action in a given state. It evaluates trade-offs between journey time, safety (e.g., collision risk), and energy consumption, thereby guiding the learning algorithm toward optimal and context-aware navigation behavior.

This formulation ensures that each AV can learn from its environment in a way that balances efficiency, safety, and resource use, leading to robust policy optimization across varied urban driving scenarios.

The framework integrates PPO and GNN to achieve the following objectives:

Problem Formulation

Let the urban environment be represented by a graph $\mathrm{G}=\left(\mathrm{V},\mathrm{E}\right)$, where V is the set of intersections in vehicles, and E is the roads connecting them. The state of the vehicle at time t is ${\mathrm{s}}_{\mathrm{t}}$ ∈ $\mathrm{t}\mathrm{o} \mathrm{S}$, the action take is ${\mathrm{a}}_{\mathrm{t}}$ ∈ $\mathrm{A}$, and the transition probability is $\mathrm{P}\left({\mathrm{s}}_{\mathrm{t}+1}\right|{\mathrm{s}}_{\mathrm{t}},{\mathrm{a}}_{\mathrm{t}})$. The reward function $\mathrm{R}\left({\mathrm{s}}_{\mathrm{t}},{\mathrm{a}}_{\mathrm{t}}\right)$ reflects the trade-offs between travel efficiency, safety, and energy consumption. The problem is modeled as MDP, aiming to maximize the cumulative reward.

$$\mathrm{J}\left(\mathsf{\pi }\right)=\mathbb{E}\left[\sum _{\mathrm{t}=0}^{\mathrm{T}}{\mathsf{\gamma }}^{\mathrm{t}}\mathrm{R}\left({\mathrm{s}}_{\mathrm{t}},{\mathrm{a}}_{\mathrm{t}}\right)\right]\dots$$

(1)

where $\mathsf{\pi }$ is the policy, T is the episode duration, and $\mathsf{\gamma }$ = (0, 1] is the discount factor.

Signal Adaptive Rout Planning for Autonomous Vehicles:

$${\mathrm{t}}_{\mathrm{s}\mathrm{i}\mathrm{g}\mathrm{n}\mathrm{a}\mathrm{l}}^{\mathrm{w}\mathrm{a}\mathrm{i}\mathrm{t}}{\mathrm{T}}_{\mathrm{m}\mathrm{a}\mathrm{x}},{\mathrm{s}}_{\mathrm{t}}\le T_{max},{\mathrm{s}}_{\mathrm{t}}\in {\mathrm{S}}_{\mathrm{s}\mathrm{i}\mathrm{g}\mathrm{n}\mathrm{a}\mathrm{l}\mathrm{s}}\dots$$

(2)

where

• ${\mathrm{t}}_{\mathrm{s}\mathrm{i}\mathrm{g}\mathrm{n}\mathrm{a}\mathrm{l}}^{\mathrm{w}\mathrm{a}\mathrm{i}\mathrm{t}}{\mathrm{s}}_{\mathrm{t}}$ is the waiting time at a signal ${\mathrm{s}}_{\mathrm{t}}$.
• ${\mathrm{T}}_{\mathrm{m}\mathrm{a}\mathrm{x}}$ is the maximum allowable delay.
• ${\mathrm{s}}_{\mathrm{t}}\in {\mathrm{S}}_{\mathrm{s}\mathrm{i}\mathrm{g}\mathrm{n}\mathrm{a}\mathrm{l}\mathrm{s}}$ only applies when the AV is at a signal state.

Obstacle Avoidance:

$$\left|\left|{\mathrm{p}}_{\mathrm{t}}-{\mathrm{p}}_{\mathrm{o}\mathrm{b}\mathrm{s}}\left(\mathrm{t}\right)\right|\right|\ge {\mathrm{d}}_{\mathrm{m}\mathrm{i}\mathrm{n}}\left({\mathrm{v}}_{\mathrm{t}}\right),{\forall }_{\mathrm{t}}{\forall }_{\mathrm{o}\mathrm{b}\mathrm{s}\mathrm{t}\mathrm{a}\mathrm{c}\mathrm{l}\mathrm{e}\mathrm{s}}{\mathrm{p}}_{\mathrm{o}\mathrm{b}\mathrm{s}}\left(\mathrm{t}\right)\dots$$

(3)

where ${\mathrm{d}}_{\mathrm{m}\mathrm{i}\mathrm{n}}\left({\mathrm{v}}_{\mathrm{t}}\right)$ is the speed-dependent safe distance.

Energy Efficiency Constraints:

$${\mathrm{E}}_{\mathrm{u}\mathrm{s}\mathrm{a}\mathrm{g}\mathrm{e}}\left(\mathrm{t}\right)\le \mathsf{\alpha }.{\mathrm{E}}_{\mathrm{m}\mathrm{a}\mathrm{x}}\dots$$

(4)

where

• ${\mathrm{E}}_{\mathrm{u}\mathrm{s}\mathrm{a}\mathrm{g}\mathrm{e}}\left(\mathrm{t}\right)$ is the energy consumed at time t.
• ${\mathrm{E}}_{\mathrm{m}\mathrm{a}\mathrm{x}}$ is the vehicle’s maximum energy budget.
• $\mathsf{\alpha }\in \left(\mathrm{0,1}\right)$ is the scaling coefficient that adjusts the energy budget threshold depending on real-time conditions (e.g., urgency, route complexity).

Road Priority constraints:

$$\sum _{\mathrm{i}\in \mathrm{P}}{\mathsf{\delta }}_{\mathrm{i}}{\mathsf{\alpha }}_{\mathrm{i}}\left(\mathrm{t}\right)\le {\mathrm{P}}_{\mathrm{m}\mathrm{a}\mathrm{x}}\dots$$

(5)

where ${\mathsf{\delta }}_{\mathrm{i}}$ represent the priority weight of road segment $\mathrm{i},\mathrm{a}\mathrm{n}\mathrm{d}{\mathrm{P}}_{\mathrm{m}\mathrm{a}\mathrm{x}}$ is the allowed maximum priority values.

3.2.1. Collision Avoidance Action Space Specification

The autonomous vehicle employs a comprehensive collision avoidance action repertoire consisting of four fundamental maneuver categories: lateral avoidance, longitudinal control, combined maneuvers, and emergency responses. The lateral avoidance actions include steering-based swerving with angular velocities ω ∈ [−0.5, 0.5] rad/s for obstacle circumnavigation, lane change maneuvers executed through sigmoid-shaped trajectory planning with lateral accelerations $a_{lat}\le 2.5{\displaystyle \frac{\mathrm{m}}{{\mathrm{s}}^2}}$, and evasive turning with maximum steering angles ${\delta }_{max}=\pm 25°$ for immediate threat response. Longitudinal control encompasses adaptive braking with deceleration rates $d\in {\displaystyle \frac{\left[0.5,8.0\right]\mathrm{m}}{{\mathrm{s}}^2}}$ based on time-to-collision calculations, speed reduction protocols maintaining minimum safe velocities $v_{min}=5{\displaystyle \frac{\mathrm{k}\mathrm{m}}{\mathrm{h}}}$ in urban scenarios, and emergency braking achieving maximum deceleration $d_{emergency}=9.5{\displaystyle \frac{\mathrm{m}}{{\mathrm{s}}^2}}$ when collision probability $P\left(C_t\right)>0.8$. Combined maneuvers integrate simultaneous steering and braking through coordinated control algorithms that optimize the trade-off between lateral stability and stopping distance, while emergency responses include complete vehicle stops, hazard signal activation, and V2X emergency broadcasts to surrounding vehicles within a 200-m radius.

3.2.2. Multi-Agent Environment Architecture

The framework operates within a comprehensive multi-agent environment where pedestrians, cyclists, and other vehicles are controlled by independent learning agents rather than scripted behaviors. Each pedestrian agent ${\pi }_{ped}$ implements a social force model with collision avoidance preferences, goal-seeking behavior toward crosswalks or destinations, and dynamic response to vehicle proximity using a safety radius $r_{safety}=2.0$ m. Vehicle agents ${\pi }_{vehicle}$ follow lane-keeping protocols, adaptive cruise control with time headway $\tau =1.5$ s, and cooperative lane-changing behaviors that communicate intentions through V2V messaging protocols. The interaction modeling employs a hierarchical game-theoretic framework where each agent optimizes its individual utility function $U_{i\left(s_t,a_t,a_{\left\{-i\right\}}\right)}$ considering both personal objectives and predicted actions of neighboring agents $a_{\left\{-i\right\}}$. Inter-agent communication occurs through a shared observation space ${\Omega }_{shared}$ containing relative positions, velocities, and intended trajectories of all agents within a 50-m perception radius, enabling proactive collision avoidance through intention prediction and cooperative path planning.

3.2.3. Travel Time Optimization

Minimizing travel time T is a primary objective for autonomous vehicle navigation in dynamic urban environments. The framework employs a hierarchical approach to travel time optimization, incorporating both baseline travel metrics and context-aware adjustments that reflect real-world navigation constraints.

Baseline Travel Time Formulation

• The total travel time for a vehicle navigating from source to destination is calculated as:

  $$T_{total}=\sum \left(k=1toN\right)\left[{\displaystyle \frac{d_k}{v_k}}+w_k\right]\dots$$

  (6)

  where
• N: Total number of discrete time steps in the journey.
• d_k: Distance traveled during time step k (meters).
• v_k: Instantaneous vehicle speed at time step k (m/s).
• w_k: Waiting time at time step k due to traffic signals, congestion, or obstacles (seconds).

Equation (6) represents the fundamental travel time calculation that captures both kinematic movement time (d_k/v_k) and stationary delays (w_k). The distance d_k is computed from the vehicle dynamics model as d_k = v_k × Δt, where Δt is the time step duration. The speed v_k is constrained by the vehicle’s physical capabilities (0 ≤ v_k ≤ v_max) and safety requirements based on local traffic conditions.

Context-Aware Effective Travel Time

To incorporate urban context awareness and dynamic traffic considerations, the effective travel time T_eff is formulated as:

$$T_{eff}=\sum \left(k=1toN\right)\left[\left({\displaystyle \frac{d_k}{v_k}}\right)\times p_k+w_k\times c_k\right]\dots$$

(7)

where

• $p_k$: Priority weight factor for road segment at time step k [dimensionless, range: 0.1–1.0];
• $c_k$: Congestion multiplier for waiting time at time step k [dimensionless, range: 1.0–3.0].

The priority weight p_k dynamically adapts based on multiple urban factors:

$$p_k=w_{traffic}\times {\rho }_k+w_{signal}\times s_k+w_{safety}\times f_k+w_{energy}\times e_k\dots$$

(8)

where

• ${\rho }_k$: Normalized traffic density factor [0.1–1.0].
• $s_k$: Signal coordination efficiency factor [0.1–1.0].
• $f_k$: Safety assessment factor [0.1–1.0].
• $e_k$: Energy efficiency factor [0.1–1.0].

  $$w_{traffic},w_{signal},w_{safety},w_{energy}:weightingcoefficients\left[sum=1.0\right]$$

  (9)

The congestion multiplier $c_k$ adjusts waiting time penalties based on traffic conditions:

$$c_k=1.0+\alpha \times \left({\rho }_k-{\rho }_{nominal}\right)+\beta \times dela{y}_{forecas{t}_k}\dots$$

(10)

where

• α = 2.0: Traffic density sensitivity parameter.
• β = 0.5: Delay forecast sensitivity parameter.
• ${\rho }_{nominal}=0.3:$ Baseline traffic density threshold.
• $dela{y}_{forecas{t}_k}:$ Predicted additional delay based on traffic patterns [seconds].

Vehicle Dynamics’ Integration in Travel Time

The travel time optimization incorporates vehicle dynamics constraints to ensure realistic performance estimates:

$$SpeedProfileOptimization:v_k^{optimal}=min\left(v_{spee{d}_{limit}},v_{traffi{c}_{flow}},v_{dynamic{s}_{max\left(a_k-1,{\delta }_k-1\right)}}\right)\dots$$

(11)

where $v_{dynamic{s}_{\mathit{max}x}}$ represents the maximum achievable speed given previous acceleration $a_k-1$ and steering angle ${\delta }_k-1$, computed from the vehicle dynamics model.

$$Acceleration-ConstrainedDistance:d_k=v_{k-1}\times \Delta t+0.5\times a_k\times \Delta t^2\dots$$

(12)

$$subjectto:\left|a_k\right|\le a_{max}and{\displaystyle \frac{\left|a_k-a_k-1\right|}{\Delta t}}\le j_{\mathit{max}x}$$

where $a_{max}=3.0{\displaystyle \frac{\mathrm{m}}{{\mathrm{s}}^2}}$ (maximum acceleration) and j_max = 2.0 m/s³ (maximum jerk).

Multi-Objective Travel Time Optimization

The framework optimizes travel time while balancing safety and energy efficiency through a weighted objective function:

$$T_{optimized}={\lambda }_{time}\times T_{eff}+{\lambda }_{safety}\times T_{safet{y}_{penalty}}+{\lambda }_{energy}\times T_{energ{y}_{penalty}}\dots$$

(13)

where

Safety Time Penalty:

$$T_{safet{y}_{penalty}}=\sum \left(k=1toN\right)\left[P\left(C_k|s_k,u_k\right)\times t_{safet{y}_{margin}}\right]\dots$$

(14)

Energy Time Penalty:

$$T_{energ{y}_{penalty}}=\sum \left(k=1toN\right)\left[{\displaystyle \frac{E_k}{E_{nominal}}}\times t_{energ{y}_{factor}}\right]\dots$$

(15)

with

• ${\lambda }_{time}$ = 0.6, λ_safety = 0.3, λ_energy = 0.1: Optimization weights.
• $t_{safet{y}_{margin}}=5.0$ s: Time penalty per unit collision probability.
• $t_{energ{y}_{factor}}=2.0$ s: Time penalty per unit energy consumption.
• $E_{nominal}$: Baseline energy consumption rate.

Adaptive Time Horizon Planning

The framework employs adaptive time horizon planning that adjusts the optimization window based on scenario complexity:

$$N_{adaptive}=N_{base}+{\Delta }_{complexity}+{\Delta }_{uncertainty}\dots$$

(16)

where

• N_base = 50: Baseline time steps (5 s at 10 Hz).
• Δ_complexity: Additional steps based on traffic density and intersection count.
• Δ_uncertainty: Additional steps based on weather and pedestrian activity.

  $${\Delta }_{complexity}=⌈10\times \left({\rho }_{avg}+0.5\times intersectio{n}_{density}\right)⌉\dots$$

  (17)

  $${\Delta }_{uncertainty}=⌈5\times \left(weathe{r}_{factor}+pedestria{n}_{density}\right)⌉\dots$$

  (18)

Real-Time Travel Time Updates

The travel time estimation is continuously updated using a recursive formulation:

$$T_{eff}^{updated}={\alpha }_{update}\times T_{eff}^{current}+\left(1-{\alpha }_{update}\right)\times T_{eff}^{predicted}\dots$$

(19)

where

• α_update = 0.7: Update rate parameter balancing stability and responsiveness.
• $T_{eff}^{current}$: Current measured travel time performance.
• $T_{eff}^{predicted}$: Predicted travel time from the optimization model.

Performance Bounds and Validation

The travel time optimization ensures realistic bounds through constraint validation:

Physical constraints:

$$Minimumtime:T_{min}={\displaystyle \frac{distanc{e}_{total}}{v_{max}\left(theoreticalminimum\right)}}$$

(20)

$$Maximumtime:T_{max}=T_{min}\times \left(1+safet{y}_{factor}+congestio{n}_{factor}\right)$$

$$Constraint:T_{min}\le T_{eff}\le T_{\mathit{max}x}$$

Validation metrics:

Prediction accuracy: ${\displaystyle \frac{\left|T_{actual}-T_{predicted}\right|}{T_{actual}}}\le 0.15$

Consistency check: Smooth temporal variation in travel time estimates.

Feasibility verification: All intermediate speeds and accelerations within vehicle limits.

This comprehensive travel time optimization framework ensures that autonomous vehicles can efficiently navigate urban environments while respecting physical constraints, safety requirements, and energy efficiency objectives. The mathematical formulation provides a robust foundation for real-time decision making that balances multiple competing objectives in dynamic traffic conditions.

Figure 7 illustrates the cumulative total travel time (T) and the effective travel time ($T_{eff}$) of an autonomous vehicle on a given road segment. The actual travel time includes dynamic values of priority assigned to roads (pt) due to congestion or signal control.

Ensuring safety is critical in urban navigation, particularly in dynamic environments with unpredictable traffic patterns and pedestrian behaviors. The DRAM employs a comprehensive probabilistic framework to evaluate collision risks and guide safe navigation decisions through Bayesian inference combined with vehicle dynamics constraints.

The red curve shows the raw cumulative travel time without priority weighting. The Max Delay = 106.0 s corresponds to the largest performance gap between actual travel under congestion/signal delays and the baseline path. This reflects absolute time lost due to congestion and traffic signals. The blue curve represents effective travel time after applying dynamic road priorities and signal weighting. The algorithm discounts delays that happen on less critical routes and emphasizes prioritized ones.Bayesian Collision Probability Assessment.

The DRAM evaluates the probability of collision P(Ct|st, ut) using Bayesian inference:

$$P\left(Ct|st,ut\right)={\displaystyle \frac{\left[P\left(st|Ct,ut\right)\times P\left(Ct|ut\right)\right]}{P\left(st|ut\right)}}\dots$$

(21)

where

• P(Ct|st, ut): Posterior probability of collision at time t given state st and action ut.
• P(st|Ct, ut): Likelihood of observing state st under a collision scenario given action ut.
• P(Ct|ut): Prior probability of collision conditioned on action ut.
• P(st|ut): Evidence probability of state st given action ut.

The posterior collision probability P(Ct|st, ut) explicitly depends on the agent’s chosen action ut, reflecting how each control decision directly affects the likelihood of an unsafe outcome. This action-dependent formulation enables the framework to evaluate the safety implications of different control strategies in real time.

Figure 8 illustrates the posterior probability of collision $P\left(C_t∣s_t,u_t\right)$ for an autonomous vehicle navigating through dynamic urban environments. The probabilities are calculated using Bayesian inference, incorporating prior probabilities, state likelihoods, and evidence probabilities over 100 time steps.

3.2.4. Energy Efficiency

Energy consumption $E$ is modeled as a function of vehicle speed $v_t$ and acceleration $a_t$:

$$E={\int }_{t=0}^{T}P\left(v_t,a_t\right)\hspace{0.17em}dt,P(v_t,a_t)=\alpha v_t^2+\beta v_t{a}_t+\gamma ,\dots$$

(22)

where

• $\alpha ,\beta ,\gamma$: Coefficients representing energy consumption characteristics.
• $a_t$: Vehicle acceleration at time t.

To ensure efficient energy usage, a constraint is imposed:

• $E\le Emax,$ where $Emax$ is the maximum allowable energy consumption.

Figure 9 represents the cumulative energy consumption (E) of an autonomous vehicle as a function of time. The plot also includes a horizontal red dashed line indicating the maximum allowable energy consumption (E max) constraint.

3.2.5. Urban Context-Aware Reward Mechanism

The reward function $R\left(s_t,u_t\right)$ dynamically adapts based on the urban context, balancing travel time, safety, and energy efficiency:

$$R\left(s_t,a_t\right)={\lambda }_1+T_{eff}^{-1}+{\lambda }_2\cdot (1-P\left(C_t∣s_t,a_t\right)-{\lambda }_3\cdot E_t$$

(23)

where

• $R(s_t,a_t)$—Reward at time step $t$ given state $s_t$ and action $a_t$.
• $T_{eff}^{-1}$—Inverse effective travel time (shorter time = higher reward).
• $P(C_t\mid s_t,a_t)$—Posterior collision probability from DRAM.
• $E_t$—Energy consumption at time $t$.
• ${\lambda }_1,{\lambda }_2,{\lambda }_3$—Weight coefficients (see

  Table 5. Parameter descriptions.

  Parameter	Description	Value/Range
  $d_t$	Distance traveled	0.1–5 km
  $v_t$	Vehicle speed	10–60 km/h
  $w_t$	Waiting time at signals	0–2 min
  $p_t$	Road priority weight	0.1–1.0
  ${\lambda }_1$	Time travel weight	0.5, 0.2, 0.1
  ${\lambda }_2$	Safety weight (highest priority)
  ${\lambda }_3$	Energy efficiency weight
  $u_t$	For travel-time weight
  ${\beta }_s$	For safety weight
  ${\gamma }_e$	For energy weight
  $Emax$	Maximum allowable energy consumption	Variable
  $T_{eff}^1$	Effective travel time considering road priority weights	Derived per Equation (7)
  $P(C_t\mid s_t,a_t)$	Posterior collision probability from DRAM	0.0–1.0
  $E_t$	Energy consumption at time $t$	J (joules)

  ).

To ensure consistency across the manuscript, the reward functions used in Equations (19) and (24) have been aligned to share the same structure. The unified reward function is now defined as a weighted combination of three key factors: effective travel time, collision probability, and energy consumption. Specifically, the reward at each time step $R_t$ is computed as the negative sum of these metrics: $R_t=-({\lambda }_1\ast T_{eff}+{\lambda }_2\ast P_{collision}+{\lambda }_3\ast E_t)$, where $T_{eff}$ is the effective travel time, $P_{collision}$ is the estimated probability of collision, and $E_t$ is the energy used.

To reflect safety as the foremost concern, the reward function assigns the highest weight ${\lambda }_2$ to the collision risk component. This ensures that policies favor safe actions even at the cost of slightly longer travel times or higher energy consumption, in alignment with real-world AV design priorities.

The weights are adjusted in real time based on traffic conditions, road priorities, and safety requirements.

Figure 10 shows the evolution of the reward function $R(s_t,u_t)$ over 100 time steps, dynamically balancing the trade-offs between travel time, safety, and energy efficiency. The reward function is calculated based on real-time adjustments to account for urban navigation constraints. The formulation prioritizes safety by rewarding lower collision probabilities $1-P\left(C_t∣s_t,a_t\right)$ while inversely penalizing effective travel time and energy consumption.

This detailed system model may guarantee a realistic representation of an urban environment so that the navigation strategies reflect safety, efficiency, and sustainability. Advanced reinforcement learning used in this paper combined with graph-based modeling allowed us to scale and generalize the results in various urban environments.

3.2.6. Perception and Object Modeling

In the proposed framework, pedestrians and other road users are modeled as dynamic obstacles with motion patterns that evolve over time. The behavior of such entities is predicted using a Kalman filter-based tracking approach, which continuously updates their estimated position and velocity based on sequential sensor readings.

The perception system employs a sensor fusion mechanism that combines data from LiDAR, RGB cameras, and radar. LiDAR provides accurate depth and spatial positioning, while cameras contribute semantic understanding such as object classification (e.g., pedestrian, vehicle, cyclist), and radar enhances velocity estimation under adverse conditions like fog or rain. This fusion ensures robustness in dynamic urban scenarios.

To estimate collision risk, a probabilistic collision model is implemented. It calculates the likelihood of collision based on the predicted trajectory overlap between the ego vehicle and detected agents. Specifically, Bayesian inference is used to determine the probability of collision at each time step, considering both historical motion data and current positional uncertainty. This risk score is then directly integrated into the reward function via the DRAM to penalize unsafe actions and encourage proactive avoidance strategies.

The action space definitions directly feed into the PPO policy optimizer within URLNF. During each decision cycle, the chosen action $u_t$ is evaluated by the UCARM, which dynamically adjusts the reward based on current traffic density, road priority, and collision probability from the DRAM. This integration ensures that both safety and efficiency considerations influence the PPO policy update in real time. Multi-agent coordination signals (e.g., predicted trajectories from surrounding vehicles) are incorporated into the GNN’s spatial-temporal state representation, enabling the PPO module to select actions that maximize long-term joint utility.

3.3. Proposed Framework

The challenges of AV navigation in complex dynamic urban scenes are complex and multi-faceted and are addressed in the proposed framework through a new hybrid deep reinforcement learning model. The framework integrates:

• Proximal Policy Optimization (PPO): When there is a need to make stable and robust policies when responding to the constraints of the surrounding environment.
• Graph Neural Networks (GNN): To model spatial-temporal relationships in the urban environment and interactions between the elements.
• Urban context-aware reward mechanism (UCARM): It adapts the reward system in line with traffic rules, real-time traffic, and accidents, and it provides contextually relevant decisions.
• DRAM: Uses Bayesian learning and MDP for risk prediction, modeling of uncertainties, and making the right decisions with regards to collision risks.

3.3.1. Proximal Policy Optimization (PPO)

PPO is used to update the navigation policy $\pi \theta (ut\mid st)$ AV, with θ being the parameter of the policy that governs the AV’s probability of selecting action $u_t$ from state $s_t$. In the following policy, the focus of PPO is not on modeling but to maximize the total reward received over time while keeping the policy stable. The optimization objective is given by:

$$J\left(\theta \right)=E_t\left[min\left(r_t\left(\theta \right)A_t,clip\left(r_t\left(\theta \right),1-ϵ,1+ϵ\right)A_t\right)\right],\dots$$

(24)

where

• $r_{t\left(\theta \right)}={\displaystyle \frac{\pi \theta \left(u_t|s_t\right)}{\pi \theta old\left(u_t|s_t\right)}}$ is the probability ratio between the updated policy and the old policy.
• $At=Q(st,ut)-V\left(st\right)$ is the advantage function, quantifying the relative benefit of action $u_t$ in state $s_t$ compared to the baseline value function $V\left(s_t\right)$.
• $ϵ$ is the clipping parameter that prevents excessive updates and ensures stable learning.

Although Equation (21) is expressed using a minimization-style formulation, it fundamentally represents a maximization objective for expected cumulative reward. In practice, this objective is optimized using gradient ascent, but for consistency with loss-based training conventions, it is framed as a clipped surrogate loss to be minimized. This formulation stabilizes updates by limiting the step size through the clipping function, which constrains the policy ratio $r_t\left(\theta \right)$ within a fixed range (typically between $1-\epsilon$ and $1+\epsilon$).

The value function $V\left(s_t\right)$ is learned using the Bellman equation:

$$V\left(s_t\right)=E{u}_t\sim \pi \theta \left[R\left(s_t,u_t\right)+\gamma V\left(s_t+1\right)\right],\dots$$

(25)

where $\gamma \in \left(\mathrm{0,1}\right]$ is the discount factor that prioritizes immediate rewards over future rewards. Figure 11 shows the workflow of PPO.

During training, the policy parameters (denoted as πθ) are updated using the PPO algorithm based on observed transitions. The framework collects batches of data containing states, actions, and corresponding rewards from environment interactions. An advantage estimation is then computed to measure how much better an action performs compared to the expected baseline. Using this, a clipped objective function is applied to stabilize the updates and prevent large shifts in policy. The gradient of this objective function is computed and used to update πθ using the Adam optimizer. A learning rate scheduler is used to reduce the learning rate as training progresses, improving convergence.

3.3.2. Graph Neural Network (GNN) for Spatial-Temporal Modeling

The urban environment is modeled as a directed graph $G=(V,E)$, where nodes $V$ represent intersections, vehicles, and pedestrian zones, and edges $E$ represent roads with attributes such as traffic density, signal states, and speed limits. Each node $v$ has an initial feature vector $h_v^0$ capturing local traffic and signal information.

Node embeddings are updated through message passing:

$$h{v}^{k+1}=\sigma \left(W^{\left(k\right)}{h}_v^{\left(k\right)}+\sum _{\left(u\in N\left(v\right)\right)}{\displaystyle \frac{1}{c_{vu}}}{W}^{\left(k\right)}{h}_u^{\left(k\right)}\right)\dots$$

(26)

where $N\left(v\right)$ is the set of neighbors of node $v$, $C_{vu}$ is a normalization factor, $W^k$ is the weight matrix for layer $k$, and $\sigma$ is an activation function (e.g., ReLU).

After $K$ layers, the final node embeddings $h_v^k$ capture spatial-temporal dependencies and are fed into the PPO policy network. The GNN parameters are optimized via backpropagation using the combined PPO policy loss and any auxiliary tasks, ensuring robust representation learning for decision making in dynamic urban environments. Figure 12 shows the Graph Neural Network (GNN) for spatial-temporal modeling.

The GNN component processes a graph representation of the environment where each node represents an entity (e.g., AV, pedestrian, traffic signal) and edges represent interactions or proximity. Each GNN layer has a learnable weight matrix Wk that transforms node embeddings during message passing. These weights are updated via backpropagation during training. The gradients are derived from the overall loss, which is computed from both the PPO policy loss and auxiliary tasks such as trajectory consistency or object classification, if enabled. The optimizer updates the GNN weights to minimize the loss, ensuring the graph representation accurately captures spatial and temporal dependencies critical for decision making.

3.3.3. Urban Context-Aware Reward Mechanism (UCARM)

To ensure that the reinforcement learning agent makes context-sensitive decisions, the proposed framework embeds real-world driving scenarios directly into the learning process. Three key urban contexts are considered: dense traffic, pedestrian interaction zones, and dynamic weather conditions. These scenarios are first quantified using physical and environmental parameters. Dense traffic is represented through vehicle density metrics, such as the number of surrounding vehicles, their average speeds, and spacing patterns—these are computed from LiDAR and radar data and incorporated as node features in the graph. Pedestrian–vehicle zones are identified using camera-based semantic segmentation, which flags areas such as crosswalks and sidewalks. These zones are assigned binary indicators and risk weights based on pedestrian activity levels. Dynamic weather conditions, including reduced visibility, slippery roads, and sensor noise, are modeled using environmental sensor readings and mapped to parameters like visibility range and road friction. These contextual features are embedded into the graph as node and edge attributes in the GNN and are also appended to the PPO agent’s state vector. This integration allows the model to learn environment-specific behaviors and adjust its reward optimization accordingly, improving its ability to generalize across varying real-world conditions while maintaining safety, energy efficiency, and travel time minimization. The reward function is defined as:

$$R\left(s_t,a_t\right)={\lambda }_1\cdot \left({\displaystyle \frac{1}{Teff}}\right)-{\lambda }_2\cdot P\left(C_t∣s_t,a_t\right)-{\lambda }_3\cdot E,\dots$$

(27)

where

• $Teff={\sum }_{t=1}^T\left({\displaystyle \frac{vt}{dt}}\cdot p_t+w_t\right)$ is the effective travel time, considering road priority weights $p_t$.
• $P\left(C_t|s_t,a_t\right)$ is the collision probability at time $t$.
• $E={\int }_{t=0}^T\left({\displaystyle \frac{1}{2\alpha vt}}+\beta v_t{a}_t+\gamma \right)\hspace{0.17em}dt$ is the energy consumption, where ${\alpha }_t$ is travel time weight, ${\beta }_s$ is safety weight, and ${\gamma }_e$ is energy weight.
• $\lambda 1,\lambda 2,\lambda 3$ are weight parameters that balance the trade-offs. Figure 12 shows the UCARM.
• The weighting parameters λ₁, λ₂, and λ₃ shown in Figure 13 are dynamically adjusted by the UCARM. This module observes external traffic and environmental conditions (e.g., congestion, pedestrian density, and weather) and tunes the reward weights accordingly to prioritize safety, efficiency, or energy optimization depending on the real-time context.

3.3.4. DRAM

The DRAM evaluates potential collision risks using Bayesian inference:

$$P\left(C_t∣s_t,a_t\right)=P\left(s_t∣C_t\right)P\left(C_t\right)P\left(s_t\right)\dots$$

(28)

where

• $P\left(C_t|s_t,a_t\right)$ is the posterior probability of collision.
• $P\left(s_t|C_t\right)$ is the likelihood of the state under a collision scenario.
• $P\left(C_t\right)$ is the prior probability of a collision.
• $P\left(s_t\right)$ is the evidence probability of the state $s_t$.

This module integrates collision probabilities into the reward mechanism, penalizing high-risk actions. Markov Decision Processes (MDPs) are used to model state transitions and evaluate policies, ensuring optimal decision making under uncertainty:

$$P\left(st+1∣st,at\right)=\sum s^{\prime }P\left(st+1∣s^{\prime },at\right)P\left(s^{\prime }∣st\right)\dots$$

(29)

To evaluate the system’s responsiveness to sudden threats, the research simulated emergency scenarios in CARLA, such as sudden pedestrian or animal crossings within close range (under 5 m). The model’s average reaction time—measured from sensor input to final control signal output—ranged between 0.152 to 0.176 s across 40 trials. This latency is within the industry-acceptable range for real-time AV systems. These results indicate that the DRAM module, when combined with UCARM, can detect and respond to high-risk scenarios in a timely manner. Further tests under high-speed conditions and low visibility environments are planned as future work. The impact of DRAM-UCARM integration on real-time responsiveness is discussed in Section 4.5.

Figure 14 shows the DRAM. The workflow of the proposed framework integrates multiple components to address the challenges of AV navigation in dynamic urban environments. The steps are as follows:

• Input Data Preprocessing: Raw data, including traffic density, signal states, road attributes, and vehicle dynamics, is collected from real-world datasets (e.g., Argoverse, nuScenes) or generated synthetically using the CARLA simulator. The data is normalized and formatted to ensure compatibility with the framework.
• Spatial-Temporal Modeling with GNN: The processed data is transformed into a graph representing nodes (e.g., intersections, vehicles) and edges (e.g., road segments). The Graph Neural Network (GNN) module captures spatial-temporal dependencies and computes node embeddings.
• Policy Optimization with PPO: The node embeddings generated by the GNN are fed into the PPO module, which iteratively refines the navigation policy πθ. This balances exploration and exploitation, ensuring robust and efficient decision-making.
• Reward Adjustment with UCARM: UCARM provides real-time changes focusing on traffic congestion, safety parameters, and energy utilization. This means that the policy will be in tune with the goals of dynamic urban navigation.
• Collision Risk Assessment with DRAM: The DRAM estimates the collision probability through the application of Bayesian analysis. This probability is incorporated in the reward function to reduce the chances of a dangerous move which reduces safety.
• Convergence and Deployment: The framework cycles through multiple episodes until it arrives at the desired state, therefore stabilizing the policy optimization phase. The trained policy is then used in real-time AV navigation to navigate the car.

Algorithm: Workflow of the Proposed Framework

Algorithm 1 outlines the step-by-step process of the proposed framework:

Algorithm 1: Workflow of the Proposed Framework (URLNF)

Input: Traffic data (e.g., density, signals, road attributes); AV state $s_t$; Initial policy ${\pi }_{\theta }$; Reward weights ${\lambda }_1,{\lambda }_2,{\lambda }_3$. Output: Optimized navigation policy ${\pi }_{\theta }\ast$. (a) Step 1: Data Preprocessing Collect raw data from real-world (Argoverse, nuScenes) or synthetic (CARLA) sources. Normalize sensor readings and structure the environment as a graph $G=(V,E)$. ○ Nodes $V$: Intersections, vehicles, pedestrian zones. ○ Edges $E$: Roads with traffic density, speed limits, signal states. (b) Step 2: Spatial-Temporal Modeling (GNN) Initialize GNN weight parameters $W_k.$ For each node $v$ in graph $G$, compute initial embedding $h_v^0$. For each GNN layer $k=1$ to $K$: ○ For each node $v$: ▪ Aggregate features from neighbors. ▪ Update embedding: $h_v^k=\mathrm{A}\mathrm{c}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}(W_k\ast Aggregate{d}_{Features})$. (c) Step 3: Policy Optimization (PPO) Initialize PPO policy parameters $\theta$ and learning rate $\eta$. Repeat until policy converges: 1. Sample trajectories: $(s_t,a_t,r_t,s_{t+1})$; 2. Compute advantage: $A_t=Q(s_t,a_t)-V\left(s_t\right)$; 3. Calculate probability ratio: $r_t\left(\theta \right)={\pi }_{\theta }\left(a_t\right|s_t)/{{\pi }_{\theta }}_{old}(a_t\left|s_t\right)$; 4. Apply PPO clipped objective: ▪ $L=min(r_t\ast A_t,clipped(r_t)\ast A_t)$; 5. Update policy: $\theta \leftarrow \theta +\eta \ast 𝛻L$. (d) Step 4: Reward Adjustment (UCARM) Dynamically adjust reward weights ${\lambda }_1,{\lambda }_2,{\lambda }_3$ based on: ○ Traffic density; ○ Pedestrian presence; ○ Current energy usage. Recompute reward: $$R_t=-({\lambda }_1\ast Effectiv{e_{Travel}}_{Time}+{\lambda }_2\ast Collisio{n}_{Probability}+{\lambda }_3\ast Energ{y}_{Consumption})$$ (e) Step 5: Collision Risk Assessment (DRAM) Estimate collision probability using Bayesian inference: $$P\left(collision\right|s_t,a_t)={\displaystyle \frac{\left[Likelihood\ast Prior\right]}{Evidence}}$$ Penalize actions with high collision risk in reward function. (f) Step 6: Convergence and Deployment Repeat Steps 3 to 5 until convergence is reached. Output the optimized policy ${\pi }_{\theta }\ast$ for real-time deployment on AVs.

3.3.5. Multi-Agent Collision Avoidance Coordination

The collision avoidance system integrates multi-agent coordination through a distributed consensus mechanism where each vehicle agent maintains a local collision avoidance policy ${\pi }_{local}$ while participating in a global coordination protocol ${\Pi }_{global}$. When multiple agents detect potential conflicts, they engage in a negotiation process using the Collision Avoidance Coordination Algorithm (CACA):

(1)

Threat assessment phase where each agent broadcasts its intended trajectory $T_{intended}$ and current risk assessment $R_{local}$.

(2)

Priority assignment based on a combination of factors including vehicle type, passenger count, and proximity to destination (emergency vehicles receive highest priority α_emergency = 1.0).

(3)

Maneuver selection where agents select from the predefined action space $\left\{swerv{e}_{left},swerv{e}_{right},brak{e}_{moderate},brak{e}_{emergency},spee{d}_{maintain},lan{e}_{chang{e}_{left}},lan{e}_{chang{e}_{right}}\right\}$ based on collective optimization.

(4)

Execution monitoring where agents continuously update their actions based on real-time feedback from other participants. The system ensures deadlock prevention through a timeout mechanism ${\tau }_{timeout}=2.0$ s, after which the agent with the highest priority executes its preferred maneuver while others adopt defensive positions.

3.3.6. Simulation Scenarios and Parameters

The proposed framework was tested in the CARLA simulator, which is a realistic simulator for testing autonomous vehicles in different urban conditions. Testing was done to assess the efficiency of this framework regarding travel time minimization, collision avoidance, and energy consumption in various cases.

The scenarios tested are as follows:

(a)

Dense Traffic Conditions: Using the simulated high vehicle density on the urban road networks, the performance of the framework was tested to determine the shortest travel time and the avoidance of collision.

(b)

Mixed Pedestrian–Vehicle Zones: Virtual environments with realistic but random pedestrian behavior, capturing safety statistics and frequency of collisions.

(c)

Dynamic Weather Conditions: Conducted the framework in various adverse environmental conditions including rain, fog, and low visibility in order to establish energy efficiency and success rates in navigation.

While the current framework utilizes reactive decision making grounded in immediate sensor inputs and real-time risk estimations, it does not yet account for future traffic evolution or potential conflicts in shared spaces like intersections. To enhance realism and scalability in future simulations, this research aims to integrate predictive behavior modeling using cooperative multi-agent reinforcement learning. This will allow AVs to exchange intent and forecast trajectories, enabling proactive adjustments before conflicts arise. For example, if a junction is predicted to be occupied by several AVs in upcoming time windows, individual agents could reroute or delay entry, reducing bottlenecks and enhancing traffic fluidity. Incorporating such cooperative intelligence is essential for navigating densely populated urban environments and can substantially reduce collision risks under heavy traffic loads.

The framework’s performance was evaluated using the following metrics:

• Travel Time: The total time taken to navigate from the source to the destination (in seconds).
• Collision Rate: This is the number of times the plane of one pilot interfaced with the plane of another pilot during a single exposition of a simulation.
• Energy Efficiency: The quantity of total power needed for the navigation (in joules).
• Navigation Success Rate: The likelihood that the system would be completed successfully, as a percentage of the total navigations done under the circumstances.

Table 6. Simulation scenarios and metrics.

Scenario	Conditions	Metrics Evaluated
Dense traffic	High vehicle density	Travel time, collision rate
Mixed pedestrian zones	Random pedestrian movements	Safety, collision rate
Dynamic weather conditions	Rain, fog, and low visibility scenarios	Energy efficiency, success rate

ensures the assessment of the proposed framework under all scenario possibilities in the different cities and reveals the flexibility of the framework within this simulation.

4. Results and Discussions

This section provides an in-depth analysis of the results obtained from the proposed methodologies evaluated on three diverse datasets: Argoverse, nuScenes, and CARLA. These datasets pertain to the actual and simulated urban environments, low- and high-mobility traffic, pedestrian and vehicle interactions, and weather fluctuations. The performance of each case is evaluated using key parameters including time taken, ability to avoid collision, and energy required in an aim to assess the scalability as well as flexibility of the framework. The results are expressed as tables and accompanied by textual annotations to account for the observed behavior and consequences of the framework in different settings. Moreover, performance of the modules is compared between the two datasets to emphasize the usefulness of each in light of the proposed framework. Every aspect of the evaluation process is connected to the characteristics of each component of the proposed system, such UCARM, DRAM, the joint PPO-GNNs, and others. This discussion also affords a theoretical analysis of the trade-offs between efficiency, safety, and sustainability in the framework and the practical payoffs of utilizing the framework. The key metrics used to evaluate the proposed framework are:

Travel Time (T): The time that was taken between the source and the destination.

Collision Rate (CR): Number of collisions that occurred in a particular simulation.

Energy Consumption (E): Total energy used for navigation in joules.

Navigation Success Rate (NSR): Proportion of successful navigation.

Reward Value (R): Last values of the reward function incorporating efficiency, safety, and energy parameters.

4.1. Results on Argoverse Dataset

The performance of the proposed framework on the Argoverse dataset, which includes realistic annotated urban traffic scenes, demonstrates the framework’s versatility. The performance is analyzed across three major scenarios, dense traffic, mixed zones, and dynamic weather, with specific measures of travel time, collision rates, energy consumption, navigation success rates, and reward values.

Table 7. Performance metrics of the proposed URLNF framework across diverse urban scenarios in the Argoverse dataset.

Metric	Dense Traffic	Mixed Zones	Dynamic Weather	Average Across Scenarios
Travel Time (s)	420 ± 20	400 ± 15	440 ± 25	420 ± 20
Collision Rate (%)	2.5	3.1	3.8	3.1
Energy Consumption (J)	12,000 ± 500	11,000 ± 450	12,500 ± 600	11,833 ± 550
Navigation Success (%)	94.2	92.5	91.0	92.6
Reward Value (R)	−8.4	−8.2	−8.6	−8.4
Braking Events (Count)	28 ± 3	32 ± 4	35 ± 5	31.7 ± 4
Acceleration Events (Count)	45 ± 5	47 ± 6	49 ± 7	47.0 ± 6
Fuel Efficiency (km/L)	14.5 ± 0.5	15.2 ± 0.6	13.8 ± 0.7	14.5 ± 0.6

presents a comprehensive evaluation of the proposed Urban Reinforcement Learning Navigation Framework (URLNF) under three urban conditions: dense traffic, pedestrian–vehicle mixed zones, and dynamic weather. Key performance indicators include average travel time, collision rate, energy usage, navigation success rate, cumulative reward, number of braking and acceleration events, and estimated fuel efficiency. The results are averaged over multiple test episodes, with standard deviations included where applicable, reflecting the model’s robustness and adaptability across varying conditions.

Figure 15 illustrates the comparative performance of the proposed URLNF framework across three distinct urban driving contexts: dense traffic, pedestrian-vehicle mixed zones, and dynamic weather. Key metrics shown include travel time, collision rate, energy consumption, navigation success rate, and fuel efficiency. The trends highlight how the framework adapts to varying levels of complexity and uncertainty, demonstrating consistent safety and efficiency outcomes.

Time taken on the road is an important measure of navigation performance. The overall average time spent traveling across the different scenarios is 420 ± 20 s, and the DW scenarios are the longest, taking 440 ± 25 s. This rise can be explained by the fact that navigation has to be done at lower speeds in adverse weather conditions like rain or fog, which is reflected in the dataset. The least travel time was recorded in mixed zones (400 ± 15,400 \pm 15,400 ± 15), thus proving the versatility of the framework in providing efficient movement within areas that have moderate traffic and pedestrian flow. The collision rates stayed below 3.1% in all the planning factors, which showed the effectiveness of the DRAM in identifying and preventing collision risks.

Dynamic weather again had the highest collision rate of 3.8%, which was attributed to the fact that changes in weather patterns were difficult to predict. However, this rate is lower than the collision rates of the conventional navigation systems, which are above 5% in comparable environments. In dense traffic, the collision rate was least to 2.5%, which proved that the model is efficient in congestion through making the right choices in real time. The energy consumption in the average of all scenarios was 11,833 ± 550 J, and the highest energy consumption was in dynamic weather—12,500 ± 600 J. This is mainly attributed to a high degree of oscillation and maneuvers such as braking and acceleration in order to achieve safe control of the vehicle under various circumstances. Mixed zones required the least energy (11,000 ± 45,011,000 \pm 45,011,000 ± 450 J) because traffic movement and pedestrian crossover activities were less likely to involve rapid acceleration or deceleration. The framework attained a favorable average navigation success rate of 92.6% in all the scenarios, with the dense traffic scenario registering 94.2%. This shows how the framework can address congestion in urban areas by using the UCARM to address safety and efficiency. Success rates were however slightly lower in dynamic weather (91.0%) because of the extreme sensitivity of the system to variations in the environment. The reward values averaged −8,4, which is the cost of the efficiency/safety/energy consumption trade-off. The lowest score was in dynamic weather, where the system is −8.6, implying that it is safer for the system to be inefficient in difficult conditions. Braking events were higher in dynamic weather (35 ± 5) than in dense traffic (28 ± 3), therefore stressing the need to respect the traffic rules while on the road. The frequency of acceleration adjustment was highest in dynamic weather (749 ± 7) as a consequence of the energy consumption, and as such, exposes the need for balance to achieve stability. Mixed zones achieved the highest fuel economy at 15.2 ± 0.6 km/L, and the lowest was achieved at 13.8 ± 0.7 km/L for the dynamic weather condition, in line with the energy consumption findings.

4.2. Results on Nuscenes Dataset

The performance of the proposed framework was tested on the nuScenes dataset, which contains multi-sensor data and complex traffic scenarios of the urban area; thus, it proves that the proposed framework is effective in different environmental and traffic conditions. The nuScenes dataset contains LIDAR, radar, and high-definition camera data, which makes it suitable for evaluating the proposed system in urban scenarios with dynamic traffic. The performance was assessed across three key scenarios, dense traffic, mixed zones, and dynamic weather, with travel time metrics, collision rates, energy consumption, navigation success rates, reward values, and other operational metrics.

Table 8. Evaluation of the proposed URLNF framework across scenarios using the nuScenes dataset.

Metric	Dense Traffic	Mixed Zones	Dynamic Weather	Average Across Scenarios
Travel Time (s)	390 ± 18	410 ± 22	430 ± 20	410 ± 20
Collision Rate (%)	1.8	2.4	3.2	2.5
Energy Consumption (J)	11,500 ± 400	12,000 ± 450	12,300 ± 500	11,933 ± 450
Navigation Success (%)	95.5	93.8	91.5	93.6
Reward Value (R)	−8.0	−7.9	−8.3	−8.1
Lane Changes (Count)	12 ± 2	15 ± 3	18 ± 3	15 ± 3
Pedestrian Interactions (Count)	8 ± 1	12 ± 2	14 ± 3	11 ± 2
Fuel Efficiency (km/L)	15.5 ± 0.6	14.8 ± 0.5	14.0 ± 0.7	14.8 ± 0.6

reports the performance of the proposed Urban Reinforcement Learning Navigation Framework (URLNF) under diverse urban conditions in the nuScenes dataset: dense traffic, mixed pedestrian–vehicle zones, and dynamic weather. Metrics include travel time, collision rate, energy consumption, navigation success, cumulative reward, lane changes, pedestrian interactions, and fuel efficiency. Results are averaged across multiple runs, with standard deviations provided to reflect variability. The framework demonstrates stable and effective navigation with high success rates and efficient energy use under varying traffic complexities.

Figure 16 visualizes the comparative performance of the URLNF framework under three urban driving conditions—dense traffic, mixed pedestrian–vehicle zones, and dynamic weather—using the nuScenes dataset. It highlights key metrics such as travel time, collision rate, navigation success, energy consumption, and pedestrian interactions. The consistent trends across scenarios demonstrate the framework’s adaptability, stability, and effectiveness in real-world-like urban environments.

The average travel time for all scenarios was 410 ± 20 s, which is considerably less than what was observed in the Argoverse dataset. This is due to the less crowded simulated scenarios in nuScenes and the improved decision making facilitated by the framework. The shortest travel time was recorded in dense traffic (18,390 ± 18 s), which explains how the proposed framework can adapt to congested road networks using real-time sensor data and the GNN for accurate spatial temporal modeling. The longest travel time was observed in dynamic weather (430 ± 20 s) because of lower speeds and additional caution in case of adverse conditions such as rain or low visibility, which are typical for the nuScenes dataset. The average collision rate was lowered considerably to 2.5%, and this was attributed to the DRAM. This is an improvement compared to the Argoverse results (3.1%). The lowest collision rate was seen in dense traffic (1.8%), where the framework could effectively manage the movement of the vehicle, which was highly predictable.

The collision rate rose to 3.2% in dynamic weather due to particularly unpredictable pedestrian and vehicle behavior in poor visibility and on slippery roads. The mean energy for all scenarios was calculated to be 11,933 ± 450 J, which is again very close to the Argoverse dataset, which was 11,833 ± 550 J. This shows that the framework can enhance the use of energy during navigation while at the same time ensuring safety. Dynamic weather required the most energy, 12,300 ± 500 J, because participants had to brake and accelerate often to navigate the course due to unfavorable weather. The lowest energy consumption was recorded in the dense traffic condition with an energy of 11,500 ± 400 J, as the framework encouraged smooth motion and minimized jerky maneuvering. The navigation success rate was the most successful of all the datasets with an average success rate of 93.6%, and dense traffic was the best with 95.5%. This shows that the framework can always hit the target in a given period and avoid traffic signals as well as any other barrier on the way. The success rate was a little lower in dynamic weather (91.5%), as conditions like low light, unpredictable pedestrian crossing, etc., posed some problems to the system. The average value of reward was −8.1. A negative value was recorded as the lowest in mixed zones (−7.9). This means that the framework was able to optimize time, safety, and energy consumption for the situations with moderate traffic and crossing occurrences. Similarly, the reward in dynamic weather was reduced to −8.3, as it takes some compromises to keep safety and stability from being hindered by unfavorable weather. The number of lane changes increased from 12 ± 2 in dense traffic to 18 ± 3 in dynamic weather, demonstrating the system’s flexibility to respond to scenarios with dynamic lane configuration. The highest values of the pedestrian interaction were in dynamic weather with 314 ± 3, which proved that the proposed framework can successfully handle pedestrians’ unpredictable behavior. As for fuel economy, the highest value was observed in dense traffic (15.5 ± 0.6 L/100 km) and the lowest in dynamic weather (14.0 ± 0.7 L/100 km), which corresponds with the energy consumption rates.

4.3. Results on CARLA Dataset

The assessment of the proposed framework on the CARLA synthetic dataset highlighted fundamental information about the system’s behavior in extreme and edge cases. CARLA has a rich and realistic environment that can mimic difficult urban conditions, including high density, segments with combined traffic pedestrian zones, and variable meteorological conditions like rain or fog. These are the scenarios that challenge the feasibility of the proposed framework most of the time, requiring dynamic adjustment of navigation systems while optimizing their efficiency, safety, and energy use.

Table 9. Performance of the proposed URLNF framework across scenarios using the CARLA dataset.

Metric	Dense Traffic	Mixed Zones	Dynamic Weather	Average Across Scenarios
Travel Time (s)	450 ± 25	440 ± 20	460 ± 30	450 ± 25
Collision Rate (%)	3.0	3.5	4.2	3.6
Energy Consumption (J)	12,500 ± 550	13,000 ± 600	13,500 ± 650	13,000 ± 600
Navigation Success (%)	92.8	91.2	89.5	91.2
Reward Value (R)	−8.7	−8.6	−8.9	−8.7
Lane Changes (Count)	20 ± 4	25 ± 5	30 ± 6	25 ± 5
Pedestrian Interactions (Count)	15 ± 3	20 ± 4	22 ± 5	19 ± 4
Fuel Efficiency (km/L)	13.5 ± 0.7	13.0 ± 0.6	12.2 ± 0.8	12.9 ± 0.7

presents a detailed evaluation of the proposed URLNF framework in simulated urban driving environments using the CARLA dataset. Results span three challenging conditions: dense traffic, mixed zones, and dynamic weather. Key performance metrics include travel time, collision rate, energy consumption, navigation success, reward value, lane changes, pedestrian interactions, and fuel efficiency. Averages and standard deviations are reported, highlighting the model’s ability to maintain safe and efficient driving under complex and varied conditions.

Figure 17 showcases the performance of the URLNF framework under three dynamic urban conditions—dense traffic, mixed pedestrian–vehicle zones, and dynamic weather—based on the CARLA simulation environment. Metrics visualized include travel time, collision rate, navigation success, energy consumption, pedestrian interactions, and fuel efficiency. The trends emphasize the framework’s robustness and adaptability in complex and high-uncertainty driving scenarios.

The overall average travel time was 450 ± 25 s, which indicated that CARLA scenarios are relatively complex compared to Argoverse (420 ± 20 s) and nuScenes (410 ± 20 s). Travel time was the highest in dynamic weather (460 ± 30 s) since it was difficult to maneuver, move at high speeds, and make several adjustments due to the weather conditions. Specifically for dense traffic, the travel time was slightly less at 450 ± 25 s, as the framework is capable of minimizing lane changing and avoiding congested regions by using GNN for spatial-temporal learning. Comparing the CARLA results with those of Argoverse and nuScenes, the collision rate was higher in CARLA (3.6%) than in Argoverse (3.1%) and nuScenes (2.5%); this indicates the challenge of avoiding risks in extreme traffic or weather conditions.

A high collision rate was found in dynamic weather (4.2%) due to uncontrolled pedestrian and vehicle movements. Nonetheless, the organization managed to control these risks through the DRAM to achieve a reasonable collision rate. The lowest collision rate was in dense traffic (3.0%), which demonstrates the superiority of the framework to predict car movements even with high traffic density. The average energy consumption was the highest of all datasets, at 13,000 ± 600 J, which is due to the need to handle more complex cases. Dynamic weather again took the highest energy consumption rate of 13,500 ± 650 J because drivers frequently had to brake and accelerate to maintain stability on slippery roads and unfavorable visibility.

Energy consumption was also slightly lower in dense traffic (12,500 ± 550 J) due to the possibility of maintaining a constant speed without frequent acceleration and deceleration. The navigation success rate was 91.2%, which is slightly lower than Argoverse (92.6%) and nuScenes (93.6) due to the complexity of CARLA scenarios. The highest success rates were achieved in dense traffic (92.8%), as the nature of the scenarios provided a structured environment for the use of the navigation policy of the framework. The lowest success rate was obtained in dynamic weather at 89.5%, showing that controlling stability and safety in unfavorable weather circumstances is a major challenge. The reward value was below average at −8.7 and the lowest in dynamic weather (−8.9) because the participants highlighted that safety is more important than time in difficult conditions. In mixed zones, the reward value was less bad (−8.6), which shows that the described framework facilitates effective safety and efficiency within the conditions that involve moderate interference from pedestrians.

The number of lane changes also rose from 20 ± 4 in dense traffic to 30 ± 6 in dynamic weather, suggesting the system’s flexibility when it comes to responding with different frequency patterns of lanes. Dynamic weather took 22 ± 5 interactions to complete the scenario, which is in line with the need to frequently update navigation paths for pedestrians. Fuel efficiency was the least among all datasets, with a mean of 12.9 ± 0.7 km/L, and the least was in dynamic weather with 12.2 ± 0.8 km/L because of much more frequent braking and accelerating.

4.4. Comparative Analysis

Table 10. Summary of AV performance across datasets.

Metric	Argoverse	nuScenes	CARLA	Best Value
Travel Time (s)	420 ± 20	410 ± 20	450 ± 25	nuScenes
Collision Rate (%)	3.1	2.5	3.6	nuScenes
Energy Consumption (J)	11,833 ± 550	11,933 ± 450	13,000 ± 600	Argoverse
Navigation Success (%)	92.6	93.6	91.2	nuScenes
Reward Value (R)	−8.4	−8.1	−8.7	nuScenes
Fuel Efficiency (km/L)	14.5 ± 0.6	14.8 ± 0.6	12.9 ± 0.7	nuScenes

Bold values indicate the best performance achieved for each metric across the three datasets. Lower values are better for Travel Time, Collision Rate, Energy Consumption, and Reward Value (less negative), while higher values are better for Navigation Success and Fuel Efficiency.

summarizes and compares the performance of the proposed Urban Reinforcement Learning Navigation Framework (URLNF) across three datasets—Argoverse, nuScenes, and CARLA. Key metrics include travel time, collision rate, energy consumption, navigation success rate, and cumulative reward. The overall average provides a holistic view of the framework’s adaptability and effectiveness across varied urban driving scenarios, demonstrating consistent performance in both real-world and simulated environments.

Figure 18 visually compares the performance of the proposed URLNF framework across three diverse datasets—Argoverse, nuScenes, and CARLA. Metrics include travel time, collision rate, energy consumption, navigation success, and reward value. The visualization highlights the consistency and adaptability of the framework across real-world and simulated environments, validating its effectiveness in handling complex and dynamic urban driving conditions.

The proposed framework has the shortest time to travel (410 ± 20 s) on nuScenes because it is densely equipped with sensors, and the traffic is well optimized for curved routes. The integration of multi-sensor data was possible to train the Graph Neural Network (GNN), which enabled us to model spatial-temporal relationships and reduce the time delay. CARLA took the longest time (450 ± 25 s) to complete the trial, and this result is consistent with the system’s performance under difficult conditions such as heavy traffic and unfavorable weather conditions. These conditions made the framework operate at low speeds, hence taking a longer time in its movements.

The mean cumulative travel time for all datasets was 427 ± 22 s, suggesting that the framework is reliable in other environments as well. To further demonstrate model generalizability, the research conducted cross-validation experiments by segmenting each dataset into distinct urban layout categories such as grid-like roads, radial layouts, and irregular topologies. The framework was then evaluated separately on each type. For example, in Argoverse, downtown scenes with high-density intersections were used, while in nuScenes, roundabouts and multi-lane avenues were isolated. The results revealed consistent performance across layout variations with a standard deviation in the navigation success rate under 1.5%. Additionally, the research validated the policy’s behavior on CARLA’s “Town01” (structured) and “Town05” (unstructured rural-like) environments to test spatial transferability. The model maintained over 90% navigation success, showing strong potential for real-world applicability.

The lowest collision rate was 2.5%, which was found in nuScenes, and the utilization of the DRAM was applicable to dense sensor configuration. The module used Bayesian inference and Markov Decision Processes (MDPs) to predict and avoid collision risks at a high level of accuracy. CARLA had the highest collision rate, equal to 3.6%, which results from such challenging conditions as unpredictable actions of pedestrians and heavy rain. Nevertheless, the collision rate remained reasonable, proving that the given framework is quite effective in handling extreme scenarios. The average collision rate when all datasets were combined was 3.1%, which was similar to or slightly better than previous studies, for example the work done by Xing et al. [33], who found a 4.2% collision rate in similar circumstances.

The overall energy consumption was highest in CARLA (13,000 ± 600 J) since the scenarios chosen were challenging and involved continuous braking and accelerating. Such actions as these were essential for safety but they raised energy consumption as a result. On the other hand, Argoverse had the least energy consumption of (11,833 ± 550 J) because its traffic flow was smooth; hence, there was no fluctuating speed and related energy outbursts.

Figure 19 illustrates the optimized navigation paths for autonomous vehicles across the three datasets: Argoverse, nuScenes, and CARLA. Both sub-plots show the AV’s motion in terms of X and Y coordinates, with the optimized path shown in the solid blue line and other potential routes in the dotted/dashed lines. The lightly shaded blue area is the “Optimized Zone,” where the vehicle is at its best in terms of navigation parameters including time to destination, energy consumption, and collision risk. From the optimized path, one sees that the pattern is much smoother, with few fluctuations than other paths, hence less time taken and less chances of collision. The other options, though reasonable, deviated just a little from the mainline; thus, they show resource consumption and safety risks. The AV shows comparable levels of navigation performance, with the optimal path lying closely within the intended course. The alternative paths are due to variations in the dynamic traffic condition of the urban environment represented in the nuScenes dataset. The CARLA environment with edge case scenarios highlight the optimized path that can easily maneuver its way around the zone. The two options show clearly distinguishable differences, while the proposed framework is crucial in handling such challenges as crowded traffic and unfavorable weather conditions. In total, the presented figure highlights the effectiveness and applicability of the proposed framework for optimizing AV navigation in various datasets. The separation of optimized and other paths demonstrates the extent to which the framework can be adjusted to environmental conditions while maintaining optimal and secure path planning.

In Figure 20, the optimized AV navigation path is evaluated using the nuScenes dataset, which presents more dynamic and cluttered urban scenes, including dense traffic, crosswalks, and mixed traffic agents. The optimized path (blue) again demonstrates clear advantages in trajectory efficiency and path stability compared to the baseline (green) and random (red) strategies. The baseline path deviates slightly due to less precise obstacle handling, while the random path shows irregular transitions. This figure confirms the effectiveness of the URLNF policy in real-world scenarios by showcasing its ability to integrate contextual parameters—such as pedestrian zones and vehicle density—into its path planning decisions, thus enhancing safety and decision robustness.

This Figure 21 illustrates the vehicle navigation paths across three datasets: Argoverse, nuScenes, and CARLA. Every subplot demonstrates the different navigation path, including the straight path (blue solid line), left turn (green dashed line), and right turn (red dotted line), to depict the efficiency of the AV’s performance at intersections and precise maneuvering. The comparison demonstrates the ability of the proposed framework to handle different urban navigation situations with a high level of accuracy. The Argoverse and nuScenes datasets showed accurate operation in real-world scenarios, while the CARLA dataset confirmed the framework’s stability in the simulated extreme conditions. These results confirm that the navigation framework is portable and flexible in various datasets and driving environments.

Figure 22 demonstrates a sophisticated multi-modal fusion architecture that addresses the critical challenge of environmental perception in autonomous vehicle navigation through a comprehensive RGB-depth-layout prediction pipeline. The framework begins with RGB imagery processing through a ResNet-50 backbone network, where convolutional layers extract hierarchical spatial features $F_{rgb}\in R^{H\times W\times 512}$ from raw camera input. Simultaneously, monocular depth estimation is performed using a dedicated depth regression network that leverages the same ResNet-50 encoder but with specialized decoder layers to produce dense depth maps $D_{pred}\in R^{H\times W\times 1}$, where depth values are normalized between 0 and 1, representing distances from 0 to 100 m. The innovation lies in the integration of prior environmental knowledge extraction, where semantic maps from training datasets are queried based on GPS coordinates and heading information to retrieve contextual priors $P_{prior}\in R^{H\times W\times C}$, where C represents the number of semantic classes, including roads, buildings, vehicles, and pedestrians. These priors are dynamically weighted based on spatial proximity and temporal consistency using a learned embedding network that maps geographic coordinates to semantic feature vectors. The self-attention fusion mechanism represents the core technical contribution, implementing a multi-head attention architecture where query matrices $Q=F_{rgb}{W}_Q$, key matrices $K=\left[F_{depth};P_{prior}\right]$ $W_K$, and value matrices $V=\left[F_{depth};P_{prior}\right]$ $W_V$ are computed through learned projection matrices. The attention weights ${\alpha }_{ij}=softmax\left({\displaystyle \frac{Q_i{K}_j^T}{\sqrt{d_k}}}\right)$ determine the importance of each depth and prior feature relative to RGB features, enabling adaptive feature selection based on scene context. The fused representation $F_{fused}={\Sigma }_j{\alpha }_{ij}{V}_j$ incorporates both geometric constraints from depth and semantic constraints from priors, creating a comprehensive environmental understanding that captures both immediate sensor observations and historical knowledge patterns. The integration into the PPO-GNN framework occurs through a carefully designed parameter extraction and embedding process. First, the fused features $F_{fused}$ are spatially pooled using learned attention pooling to create compact representations suitable for policy networks: ${\theta }_{layout}=GlobalAttentionPool\left(F_{fused}\right)$. Second, these layout embeddings are incorporated into the GNN architecture by augmenting node features h_v with environmental context: $h_v^{enhanced}=\left[h_v^{original};{\theta }_{layout};spatia{l}_{encoding\left(v\right)}\right]$. The GNN message passing then propagates this enriched information: $m_{uv}=MLP\left(\left[h_u^{enhanced};h_v^{enhanced};edg{e}_{feature{s}_{uv}}\right]\right)$, followed by node updates ($h_v^{l+1}=GRU(h_v^l$,${\Sigma }_u\in N\left(v\right)m_{uv})$. Finally, the PPO policy network receives the enhanced graph representation $G_{enhanced}$ along with traditional state observations $s_t$ to compute action probabilities: ${\pi }_{\theta \left(a_t|s_t,G_{enhanced}\right)}=softmax\left(PolicyMLP\left(\left[s_t;ReadoutGNN\left(G_{enhanced}\right);{\theta }_{layout}\right]\right)\right)$. The cross-dataset validation demonstrates the framework’s generalizability across CARLA (synthetic), nuScenes (real-world highway), and Argoverse (urban intersection) environments. Evaluation metrics include Intersection over Union (IoU) for semantic segmentation accuracy, pixel-wise accuracy for spatial precision, and downstream task performance measured through navigation success rates and collision avoidance effectiveness. The results show IoU scores of 0.87 (CARLA), 0.91 (nuScenes), and 0.84 (Argoverse), indicating robust cross-domain performance despite varying data distributions, lighting conditions, and scene complexities. The depth estimation achieves mean absolute errors of 0.12 m, 0.08 m, and 0.15 m, respectively, while the integrated PPO-GNN framework demonstrates 15% improved navigation performance compared to baseline methods that lack prior environmental integration. This comprehensive evaluation validates that the self-attention fusion mechanism successfully bridges the domain gap between datasets while maintaining the semantic consistency required for reliable autonomous navigation in diverse urban environments.

Both datasets show how the system is able to leverage raw RGB inputs with no prior knowledge of depth, estimate depth, forecast the layout of the environment, and determine efficiency by comparing the predicted layout with actual layout of the environment. Such an analysis also validates real-time RGB-DEP and layout estimation on CARLA synthetic, Argoverse, and nuScenes real environments from the proposed framework. The reliability of CARLA and its dataset under the worst case has been demonstrated, while Argoverse together with nuScenes made sure that the proposed framework could tackle real-world challenges. The results presented in this paper demonstrate that the discussed framework can generate repeatable and stable predictions when applied to various aspects of urban environments.

The outcomes that have been attained are typically less than in other similar works: for example, Liu et al. [17] estimate that average energy consumption for level 4–5 automated cars in kinetic populated city conditions becomes 13,500 J, but the provided framework was more effective by 5.9%. The navigation success rate was 93.6% in nuScenes and 91.2% in CARLA because of the challenging environment of the later. Argoverse’s performance obtained 92.6% success, which demonstrates that the proposed framework can perform efficiently in moderately complex real-world scenarios.

The overall success rate of 92.5% is higher than the success rate in other studies like Samma et al. [23], where the success rate of reinforcement learning-based navigation systems was 90.1% under the same circumstances.

The reward values were almost similar for two datasets, but CARLA had the lower value (−8.7) and nuScenes had higher value (−8.1). These results suggest that the framework can find good solutions that minimize travel time, safety risks, and energy consumption in a reasonable amount of time, even in the worst case.

To further evaluate the framework, its performance was compared against three previous studies shown in

Table 11. Comparison with previous studies.

Metric	Proposed Framework	Xing et al. [33]	Liu et al. [17]	Samma et al. [23]	Bilban and İnan [37]
Travel time (s)	427 ± 22	440 ± 25	450 ± 30	435 ± 28	460 ± 25
Collision rate (%)	3.1	4.2	3.8	4.0	3.4
Energy consumption (J)	12,222 ± 533	13,500	13,000	12,900	13,200
Navigation success (%)	92.5	90.5	91.0	90.1	91.8
Core method	PPO + GNN + UCARM + DRAM	DRL + MPC	Adaptive speed DRL	Improved DRL visual navigation	Improved PPO
Key differentiator	Spatial-temporal modeling, dynamic reward and risk modules	Coordinated platoon control	Speed optimization under real-time conditions	Visual adaptability in UAV and AV navigation	PPO tuned for stability, no GNN or contextual modules

.

In addition to the methods summarized in Table 11, we compared our framework with the recent work of Bilban and Inan [37], who proposed an improved PPO algorithm for AV navigation in CARLA simulations. Their approach achieved a travel time of approximately 460 s and a collision rate of 3.4%, with notable stability improvements through hyperparameter tuning. In contrast, URLNF achieved a shorter travel time (450 ± 25 s) while integrating GNN-based spatial-temporal modeling, UCARM for context-aware rewards, and DRAM for proactive risk assessment. While the collision rate was similar (3.6%), our approach demonstrated superior adaptability across multiple datasets, suggesting improved generalization potential beyond CARLA scenarios.

The proposed framework adopted for this work incurred 3% less travel time on average than the other approaches used from other literature such as Xing et al. [33] and Liu et al. [17]. The above improvement is mainly due to the successful application of the route optimization using GNN and PPO. All the collision rates proposed below are less than those of the previous studies, the average of which obtained an improvement of 26.2 percent as compared to that of Xing et al. [33]. This is a clear indication that the DRAM is better placed in tackling collision risks. The framework’s energy efficiency was modeled as having 9.5% lower power than in Liu et al. [17]. This was made achievable by UCARM to discourage wastage and promote efficiency. Thus, the proposed framework had a 92.5% navigation success rate, which is better than Samma et al. [23] by 2.4%, thus highlighting the usefulness of the framework in several urban scenarios. Comparing the proposed framework with state-of-the-art methods reveals that the impact rates of the proposed framework are greater than the impact rates of the state-of-the-art methods by all crucial measures. That this model can decrease the traveling time, mitigate collision issues, optimize energy consumption, and show satisfactory levels of navigation achievement makes it relevant for real and simulated environments. The obtained results prove the effectiveness of the presented approach based on the union of concepts in reinforcement learning and the spatial-temporal graph-based model for autonomy of car movement. Figure 23 shows the comparison with previous studies.

In addition to learning-based RL baselines, we compared the proposed URLNF framework against traditional rule-based controllers—specifically, the Intelligent Driver Model (IDM) for longitudinal control and MOBIL for lane changing. As shown in

Table 12. Performance comparison of URLNF and traditional rule-based controllers in dynamic urban environments.

Controller	Travel Time (s)	Collision Rate (%)	Navigation Success Rate (%)
IDM + MOBIL	470 ± 25	5.8	85.4
URLNF (Proposed)	427 ± 22	3.1	92.5

, IDM + MOBIL achieved a navigation success rate of 85.4% with a collision rate of 5.8%, whereas URLNF improved the success rate by over 7% and reduced collisions by nearly half. This performance gap is attributed to URLNF’s spatial-temporal modeling and context-aware decision making, enabling adaptive responses to dynamic urban traffic that static rule-based approaches cannot match.

Comparative Evaluation with Existing RL Models

To address the concern regarding novelty and provide empirical evidence of the proposed framework’s superiority, the research conducted a baseline comparison against standard deep reinforcement learning models: Deep Q-Networks (DQNs) and vanilla PPO without GNN integration. These models were trained under the same environmental settings using Argoverse, nuScenes, and CARLA datasets for uniformity in evaluation.

Table 13. Comparative evaluation of URLNF vs. existing RL methods (average metrics).

Model	Travel Time (s)	Collision Rate (%)	Reward Value (R)	Fuel Efficiency (km/L)
DQN	460 ± 30	4.8	−9.2	13.2 ± 0.8
PPO	435 ± 25	3.9	−8.7	13.9 ± 0.6
URLNF	420 ± 20	3.1	−8.4	4.5 ± 0.6

presents the averaged performance metrics across all scenarios. The results clearly demonstrate that URLNF outperforms both DQN and vanilla PPO in terms of travel time efficiency, collision reduction, and fuel consumption. The inclusion of Graph Neural Networks and the urban context-aware reward mechanism contribute significantly to the improved outcomes.

4.5. Ablation Study and Real-World Applicability

An ablation study was conducted to isolate the contributions of UCARM and DRAM. Removing DRAM increased collision rates by 25% relative to the full model, while removing UCARM reduced success rates and increased travel time. This confirms that both modules are essential for achieving optimal performance. Additionally, while CARLA simulations provide diverse urban scenarios, the absence of sensor-noise modeling may overstate real-world applicability. Future work will incorporate perception noise to validate safety thresholds under realistic urban sensing conditions.

The ablation study (

Table 14. Ablation study results showing the individual impact of UCARM and DRAM modules on URLNF performance in dynamic urban navigation.

Framework Variant	Travel Time (s)	Collision Rate (%)	Success Rate (%)
URLNF (full)	427 ± 22	3.1	92.5
URLNF—UCARM only	432 ± 24	3.5	90.8
URLNF—DRAM only	435 ± 25	3.9	90.2

) isolates the contributions of UCARM and DRAM to URLNF’s overall performance. The full URLNF achieved optimal results—427 ± 22 s for travel time, a 3.1% collision rate, and a 92.5% success rate—showing the synergy of both modules. Removing UCARM increased travel time and reduced success rate, confirming its role in travel efficiency and compliance with traffic rules. Removing DRAM raised the collision rate and further lowered success rate, highlighting its critical function in real-time risk mitigation. These results validate that both modules are essential for achieving superior safety and efficiency in dynamic urban navigation, directly differentiating URLNF’s hybrid approach from baseline models.

4.6. Impact of Look-Ahead Horizon Length

This evaluates how varying the look-ahead planning horizon affects navigation performance in dense urban environments. The horizon length determines how far ahead the framework predicts and optimizes its trajectory at 10 Hz sampling. We compare success rates, travel times, and collision rates for 2 s, 5 s (baseline), and 10 s horizons.

Table 15. Sensitivity of navigation success rate to look-ahead horizon length.

Horizon Length	Steps @ 10 Hz	Navigation Success Rate (%)	Average Travel Time (s)	Collision Rate (%)
2 s	20	86.8 ± 1.2	438 ± 24	4.6
5 s (baseline)	50	92.5 ± 0.8	427 ± 22	3.1
10 s	100	91.4 ± 1.0	430 ± 23	3.3

reports navigation success rate, average travel time, and collision rate for three horizon settings (2 s, 5 s, 10 s) at 10 Hz. Results are averaged over 10 runs per configuration with standard deviations for success rates. The baseline 5 s horizon provides the optimal trade-off between foresight and adaptability, yielding the highest success rate and lowest collision rate among the tested configurations.

Figure 24 shows that the 5 s horizon yields the highest navigation success rate, avoiding short-horizon deadlocks and long-horizon prediction errors.

Figure 25 indicates that average travel time is minimized at the 5 s horizon, with longer or shorter horizons slightly increasing trip duration.

Figure 26 reveals that collision rate is lowest at 5 s, confirming it as the optimal trade-off between foresight and real-time adaptability in dense urban navigation.

4.7. Collision Avoidance Action Analysis

Detailed analysis of collision avoidance maneuvers reveals distinct behavioral patterns across different threat scenarios. Lateral avoidance maneuvers comprised 34.2% of all collision avoidance actions in Argoverse, 41.7% in nuScenes, and 28.9% in CARLA, with average swerving angles of 12.3°, 8.7°, and 15.1°, respectively. Longitudinal control actions represented 45.1%, 38.2%, and 52.3% of responses, featuring mean deceleration rates of 4.2 m/s², 3.8 m/s², and 5.1 m/s² during moderate braking scenarios. Emergency braking events (deceleration > 7.0 m/s²) occurred in 8.3%, 6.1%, and 12.8% of collision scenarios, with reaction times averaging 0.156 s, 0.142 s, and 0.173 s from threat detection to brake engagement. Combined maneuvers integrating simultaneous steering and braking achieved the highest success rates (96.8%, 97.2%, and 94.5%) but required 23% longer execution times due to stability considerations. Lane change maneuvers demonstrated completion rates of 92.1%, 94.8%, and 89.3% with average lateral displacement distances of 3.2 m, 2.9 m, and 3.6 m while maintaining longitudinal speeds within 15% of target velocities.

Figure 27 demonstrates a sophisticated collision avoidance analysis framework validated across three diverse autonomous vehicle datasets, providing empirical evidence of specific maneuver execution and effectiveness quantification that directly addresses concerns about vehicle navigation being limited to straight-line trajectories. The Argoverse dataset analysis (left panel) showcases urban intersection navigation with measured performance metrics, including a 0.156 s reaction time, 12.3-degree swerving angle, and 96.8% overall success rate, where the ground truth path (solid blue line) clearly demonstrates non-linear trajectory execution around red circular obstacles representing collision threats. The vehicle successfully executes a sequence of collision avoidance actions marked by gold star indicators: initial threat detection at coordinates (1.0, 0.8) followed by a swerving maneuver at (1.5, 1.5) involving a 12.3-degree steering angle deviation from the baseline trajectory, pedestrian avoidance through speed reduction at (2.5, 2.5), and path resumption at (3.0, 3.0), with each action demonstrating measurable deviations from straight-line motion that validate the framework’s capability to execute complex evasive maneuvers. The nuScenes dataset validation (center panel) focuses on highway merging scenarios with enhanced performance metrics showing a 0.142 s reaction time, 8.7-degree swerving angle, and 97.2% success rate, where the predicted path (red dashed line) closely follows the ground truth while maintaining safe distances from obstacles through dynamic lane change maneuvers. The collision avoidance sequence includes lane change initiation at (1.5, 1.0) in response to a slow-moving vehicle obstacle, cooperative merging behavior at (2.5, 2.0) that demonstrates multi-agent coordination capabilities, and final lane confirmation at (3.5, 3.0), with the predicted reinforcement learning path achieving 97.2% alignment with optimal collision avoidance trajectories compared to 73.1% for baseline methods that fail to adequately respond to dynamic traffic conditions. The CARLA dataset analysis (right panel) presents the most challenging edge-case scenarios with a 0.173 s reaction time, 15.1-degree maximum swerving angle, and 94.5% success rate under extreme complexity conditions, where the vehicle must navigate multiple simultaneous collision threats including sudden pedestrian crossings, erratic vehicle behavior, and complex urban obstacles. The technical implementation reveals that collision avoidance actions are not predetermined scripted responses but rather learned behaviors emerging from the integration of the UCARM and DRAM, where Bayesian inference continuously updates collision probability estimates $P\left(C_t\right|s_t,a_t)$ and triggers appropriate maneuver selection from the action space, including lateral avoidance (swerving), longitudinal control (braking), emergency responses, and combined maneuvers. The effectiveness zones surrounding each obstacle (depicted as light red circular areas) demonstrate the spatial regions where different collision avoidance strategies are optimally deployed, with high-effectiveness zones (closer to optimal paths) achieving 95–98% success rates while challenging zones (requiring more complex maneuvers) maintain 87–92% effectiveness, thereby providing quantitative validation that the autonomous vehicle framework successfully executes sophisticated collision avoidance maneuvers with measurable performance characteristics that substantially exceed straight-line navigation capabilities and demonstrate real-world applicability across diverse urban traffic scenarios. Figure 27 shows the Collision avoidance action analysis.

4.8. Multi-Agent Collision Avoidance Performance

The multi-agent environment validation demonstrates sophisticated coordination capabilities across 1200 simulated scenarios involving two to eight simultaneous agents. Pedestrian–vehicle interactions showed successful avoidance in 97.3% of cases, with pedestrians exhibiting predictive behavior by initiating evasive actions 2.1 s before potential collision, while vehicles maintained safe distances averaging 2.8 m during close encounters. Vehicle–vehicle coordination achieved 94.7% conflict resolution without external intervention, with agents successfully negotiating lane changes, merge scenarios, and intersection priorities through the distributed consensus mechanism. Communication effectiveness measured through V2X message success rates averaged 98.2% within the 50 m perception radius, with message latencies of 12.4 ms enabling real-time coordination. Priority-based resolution demonstrated clear hierarchical behavior: emergency vehicles achieved 100% priority compliance, passenger vehicles with higher occupancy received preference in 89.3% of conflicts, and commercial vehicles successfully coordinated in 91.8% of freight corridor scenarios. Deadlock prevention mechanisms activated in only 0.7% of multi-agent scenarios, with timeout-based resolution successfully preventing system stalls. The framework’s ability to handle complex multi-agent scenarios while maintaining individual agent autonomy validates the effectiveness of the distributed collision avoidance architecture in realistic urban traffic conditions. Figure 28 shows the multi-agent collision avoidance performance.

4.9. Collision Avoidance Maneuver Effectiveness

Quantitative analysis of individual collision avoidance maneuvers reveals performance variations based on scenario complexity and environmental conditions. Swerving maneuvers achieved success rates of 94.2% (simple obstacles), 89.7% (complex intersections), and 86.1% (dense traffic), with optimal swerving angles ranging from 8–15° depending on vehicle speed and available lateral space. Emergency braking demonstrated 98.3% effectiveness in preventing collisions when activated with a time to collision ≥ 1.2 s, degrading to 76.4% effectiveness when reaction time fell below 0.8 s. Lane change maneuvers showed dependency on traffic density, achieving 96.1% success in light traffic (≤15 vehicles/km), 88.7% in moderate traffic (15–30 vehicles/km), and 72.3% in dense traffic (>30 vehicles/km). Combined steering-braking maneuvers required an average of 3.4 s for completion but achieved the highest overall safety margins, maintaining minimum distances of 1.8 m from obstacles compared to 1.2 m for single-action responses. Speed adjustment strategies proved most effective in highway scenarios (94.8% success) compared to urban intersections (81.2% success), with optimal deceleration profiles following exponential decay functions that minimize passenger discomfort while ensuring collision avoidance. Figure 29 shows the collision Avoidance Maneuver Effectiveness.

4.10. Discussions

As shown by the results of this work, the proposed framework meets the goal of AV navigation optimization in terms of the total travel time, safety, and energy consumption under various datasets and sophisticated scenarios. The specific one, the urban context-aware reward mechanism (UCARM), was very useful in proceeding with the proposed adaptation of the greedy algorithm to the dynamic structure and behavior of the urban environment, including real-time traffic information, road priority, and safety. It was this approach that made it possible to make navigation decisions with regard to context and at the same time minimize time used on the road where it was not necessary, in the process avoiding any extra fuel consumption or increased probability of an accident. DRAM was useful in reducing collision risks through its involvement of Bayesian inference and Markov Decision Processes. Through the correct estimation of possible collisions and the punishment of risky actions, DRAM helped to further improve the performance of the system during different weather conditions and dense traffic. The framework was able to prove its solidity and expansiveness by showing similar performance across the real-world datasets of Argoverse or nuScenes as well as synthetically designed edge-case datasets in CARLA. In real-world datasets, the framework was found to perform exceptionally in improving navigation by leveraging sensor-rich data to enable dynamic environment sensing. In the more difficult synthetic environments, it demonstrated its capacity to operate in extreme conditions where safety is the paramount concern but efficiency is still reasonable. The results support the proposed framework’s efficiency and will be expanded in future work to improve its applicability to unstructured and highly dynamic contexts. If the real-time feedback mechanisms were integrated to the framework, the decision-making process could be enriched during the interaction, and the unforeseen situations may be handled more effectively. Moreover, expanding the model’s reach by covering different road networks, different types of vehicles, and different cultural traffic rules will make a broader contribution towards real-world self-driving navigation systems.

To assess the system’s responsiveness in critical real-time scenarios, this research conducted a series of emergency simulations in the CARLA environment. These scenarios involved sudden appearances of pedestrians, animals, or cyclists crossing the vehicle’s path at close range (typically within 3–5 m), simulating urban uncertainties such as jaywalking or abrupt intrusions. The system’s reaction time was measured as the total time elapsed from the moment the threat was detected by onboard sensors to the execution of a corresponding control signal by the policy module.

Across 40 randomized trials, the model demonstrated an average reaction time between 0.152 and 0.176 s, depending on the complexity of the scene and proximity of the intruding object. This latency falls well within the commonly accepted benchmark (under 0.2 s) for safe, real-time operation in autonomous driving systems. The responsiveness is largely attributed to the integration of the DRAM with the UCARM, which together enable the system to evaluate collision probabilities and adapt actions dynamically.

These findings suggest that the proposed framework is capable of timely evasive actions in high-risk, unpredictable environments. Nevertheless, additional evaluations are planned under more challenging conditions, such as high-speed driving, poor lighting, and adverse weather, to validate the robustness of the system in real-world deployment scenarios.

To assess scalability, we extended URLNF to a scenario with up to 10 concurrent AV agents in CARLA. The framework maintained above a 90% success rate for up to five AVs, with a linear increase in computation time per step (~6 ms/vehicle). Beyond seven AVs, success rates dropped due to inter-agent coordination complexity, indicating a need for explicit multi-agent communication protocols in future work.

4.11. Limitations and Future Work

While the URLNF (Urban Reinforcement Learning Navigation Framework) demonstrates robust performance in dynamic urban environments using structured datasets like Argoverse, nuScenes, and CARLA, several limitations remain that impact its generalizability to real-world deployment scenarios.

Firstly, the current experiments are conducted within simulation platforms that, despite their sophistication, cannot fully replicate the diversity and unpredictability of real-world traffic. The tested scenarios are primarily based on predefined traffic flows and constrained event variations. This excludes a wide range of uncertainties such as unplanned road closures, spontaneous pedestrian crossings, inconsistent driver behaviors, or rapid weather changes, all of which are commonly encountered in real city environments.

Secondly, the sensory input used in the framework assumes near-perfect functioning of LiDAR, cameras, and radar modules. In practical deployment, these sensors can be affected by occlusions, heavy rain, snow, dirt, or lighting conditions. For instance, a fogged-up camera or LiDAR reflection in rain may lead to degraded perception. Such degradation is not currently modeled, making the perception module potentially fragile under harsh or unpredictable environments.

Moreover, the collision risk assessment module assumes a certain behavior distribution from surrounding agents, but in reality, human drivers may violate traffic rules or exhibit irrational maneuvers that are difficult to predict through standard models. Similarly, the energy optimization module operates under uniform battery performance assumptions without accounting for variable loads, terrain changes, or thermal effects on energy consumption.

In terms of data limitations, the Argoverse and nuScenes datasets primarily represent US-based and Asian urban layouts, respectively. These may not fully reflect traffic dynamics in other regions with different road infrastructures, cultural driving behaviors, or traffic control systems.

To address these gaps, future work will focus on the following directions:

• Uncertainty Modeling: Incorporate robustness-aware reinforcement learning techniques to account for noisy sensor data, missing information, or false detections.
• Adversarial Testing: Simulate aggressive or rule-violating agents to challenge the model’s decision-making robustness and adaptiveness in unexpected scenarios.
• Weather and Environmental Variability: Integrate environmental perturbation models that emulate rain, fog, glare, and occlusions into the simulation loop.
• Real-World Validation: Plan controlled pilot testing in small-scale real environments (e.g., closed campus roads or parking lots) to evaluate policy adaptation and perception reliability in live settings.
• Cross-Domain Dataset Expansion: Extend training and evaluation to include datasets from varied geographies, driving cultures, and infrastructural layouts to ensure broader applicability and generalization.

These enhancements will help in evolving the current framework into a production-grade autonomous vehicle navigation system capable of safe, efficient, and adaptive operation across diverse urban environments.

5. Conclusions

This paper introduces a new method for the path planning of autonomous vehicles that can successfully solve the problems of the dynamic environment of cities. The use of both UCARM and DRAM has been important in achieving a good balance of travel time, safety, and energy optimization. UCARM responds to real-time traffic situations and ensures safety and environmentally friendly navigation, while DRAM, through Bayesian inference and MDPs, estimates and avoids collision risks. The framework’s robustness and scalability were validated across three diverse datasets: Argoverse, nuScenes, and CARLA datasets. When tested on Argoverse and nuScenes, which are real-world scenarios, the framework proved to perform better by effectively exploiting sensor-rich data to navigate through traffic patterns. In CARLA, the framework was easily able to handle extreme conditions like dense traffic, adverse weather, and unpredictable behaviors of pedestrians as a part of edge cases. The findings show reduced travel time and collision rates higher than those of previous studies as well as constant energy efficiency. The success of this system can be attributed to the use of PPO and GNN for the modeling of spatial-temporal relationships that were critical to the decision-making context of the traffic management system. However, the framework suggested by the study creates new directions for future research. Future work will concern its ability to better operate in unstructured and highly dynamic environments like rural roads or mixed-traffic systems. The use of feedback mechanisms for real-time learning will enable the framework to adapt well to emergent situations. Moreover, the generalization of the model that includes various types of roads, local culture for traffic, and interactions of multiple agents will expand its real-world uses. This work provides a solid base for the creation of large-scale, robust autonomous navigation systems that are ready for practical utilization.

Future work can explore integrating fuzzy-based decision systems and metaheuristic optimization techniques with reinforcement learning to further enhance adaptability in highly dynamic or uncertain urban environments. Fuzzy logic can help AVs handle ambiguous scenarios, such as interpreting pedestrian intent or negotiating right-of-way in unregulated intersections, while metaheuristic algorithms like Genetic Algorithms, PSO, or Ant Colony Optimization can be used to optimize hyperparameters, reward weights, or route selections in real time. Moreover, deployment in real-world AV testbeds equipped with V2X communication infrastructure will be critical to evaluate system performance under actual traffic conditions. Implementing the proposed framework on edge devices (e.g., Nvidia Jetson, Qualcomm AI platforms) will allow low-latency decision making directly onboard AVs, supporting faster reactions in dense traffic. Additionally, future research should focus on expanding inter-agent communication protocols, enabling cooperative decision making between AVs and infrastructure. Considering cultural-specific driving behaviors (e.g., lane discipline, overtaking habits) will also improve model realism, generalizability, and policy transfer across different countries and urban settings.