Advancements in Embedded Vision Systems for Automotive: A Comprehensive Study on Detection and Recognition Techniques

Abstract

Embedded vision systems play a crucial role in the advancement of intelligent transportation by supporting real-time perception tasks such as traffic sign recognition and lane detection. Despite significant progress, their performance remains sensitive to environmental variability, computational constraints, and scene complexity. This review examines the current state of the art in embedded vision approaches used for the detection and classification of traffic signs and lane markings. The literature is structured around three main stages, localization, detection, and recognition, highlighting how visual features like color, geometry, and road edges are processed through both traditional and learning-based methods. A major contribution of this work is the introduction of a practical taxonomy that organizes recognition techniques according to their computational load and real-time applicability in embedded contexts. In addition, the paper presents a critical synthesis of existing limitations, with attention to sensor fusion challenges, dataset diversity, and deployment in real-world conditions. By adopting the SALSA methodology, the review follows a transparent and systematic selection process, ensuring reproducibility and clarity. The study concludes by identifying specific research directions aimed at improving the robustness, scalability, and interpretability of embedded vision systems. These contributions position the review as a structured reference for researchers working on intelligent driving technologies and next-generation driver assistance systems. The findings are expected to inform future implementations of embedded vision systems in real-world driving environments.

1. Introduction

Road safety has been growing in the last few years, due to road traffic that can be interpreted as exponentially evolving [1,2]. The thing that can guarantee the safety of all road users is the proposal of an intelligent in-vehicle system (ITS) that can be feasibly used in the future for Smart Cities [3,4]. Smart roads also play a key role by integrating technologies that support automation, connectivity, and energy-efficient traffic management. Nonetheless, recent studies [5] have highlighted existing uncertainties regarding their real impact on road safety and traffic operations.

This system has the objective of identifying and recognizing traffic road signs, as well as road edges, to assist the vehicle’s driver, as this is a major problem in the analysis of road scenes [6]. Several driver assistance systems have been proposed and implemented, and until now, research has continued to optimize traffic sign detection and recognition [7], as well as road lane detection [8]. Most approaches can be divided into three phases: localization, detection, and recognition (classification) [9,10,11,12]. Apparently, the first two phases are complementary, and most authors summarize them into only two categories: detection [13] and recognition for traffic signs [14,15] and road lanes [16,17,18]. The detection phase is very important, because it is used to detect all the signs and lanes of the road in order to eliminate as many objects as possible that may appear in the image, allowing us to reduce the false positive rate. Even if there is false detection in the detection phase, the performance will be corrected in the recognition phase, as this is performed in an intelligent way, using artificial intelligence for signs [19,20,21,22,23] and lanes [24,25,26].

Intelligent Transport Systems (ITSs) play a crucial role in the evolution of modern transportation, integrating advanced technologies to enhance safety, efficiency, and user comfort. For example, one study explored research and development programs for intelligent vehicles across the globe, highlighting emerging trends in this field. Another paper [27] explored the application of intelligent systems to optimize urban mobility and security through IoT-based innovations. It proposed models for smart traffic lights, parking systems, and vehicle theft detection, leveraging technologies like sensors, Raspberry Pi processors, and cloud databases. These systems dynamically adjust traffic signals based on density, monitor parking space availability, and provide real-time theft alerts. The research highlights the role of intelligent systems in improving traffic flow, enhancing parking experiences, and ensuring vehicle security. This research [28] investigates the role of unmanned aerial vehicles (UAVs) in optimizing the movement of emergency medical vehicles in Smart Cities. Using a hybrid Cascade-ResNet model, the study analyzed video data to detect congestion and unusual activity, facilitating faster and more efficient route planning. UAVs, functioning as IoT edge devices, enhanced traffic management and public safety through real-time insights. The approach demonstrated a 2.5% improvement in accuracy over comparable methods, showcasing its practical utility in urban emergency response systems. Moreover, research on embedded systems for autonomous vehicles has emphasized the importance of artificial intelligence in real-time decision-making [29], enabling vehicles to dynamically analyze their environment and the context in which they operate. These technological advancements highlight the growing importance of ITSs in developing safer and smarter vehicles [30], addressing the challenges of modern urban environments.

The evolution of visual perception in autonomous vehicles has progressed from rule-based methods and handcrafted features (e.g., Hough Transform, SVM) to modern deep learning architectures such as CNNs and Vision Transformers. Each generation has improved accuracy, generalizations, and robustness in complex road environments. Recent developments include end-to-end models that integrate perception, reasoning, and control in unified frameworks. This historical trajectory underscores the growing importance of AI-driven vision in automotive systems.

This paper undertakes an in-depth review of embedded vision systems tailored to the detection and recognition of road signs and lane markings, which are critical for enhancing road safety. By analyzing existing studies, it highlights both the strengths and limitations of current approaches, with a focus on their practical implementation under diverse conditions. The primary aim is to underscore the role of these systems in reducing accidents and improving traffic flow while addressing challenges such as environmental variability and computational efficiency. To add clarity and originality, the review intends to identify research gaps and suggest potential advancements, positioning itself as a resource for developing more effective vision-based solutions in modern transportation systems. Including a distinct problem statement in the Introduction further emphasizes the paper’s objectives and its contribution to the field. Also, a comprehensive summary of modern, real-time techniques utilized for both detecting and classifying traffic signs and road markings is provided. In this discourse, we delineate the methodologies for locating and detecting road signage and boundaries into a tripartite framework:

• Colorimetric Techniques: This division includes strategies that implement segmentation processes for the purpose of localization.
• Geometrical Approaches: This segment comprises methods that utilize geometric principles.
• Learning-Based Algorithms: This category encompasses techniques that incorporate learning paradigms for identification and classification tasks.

The categorization of recognition methodologies within this study bifurcates into two distinct streams:

• Feature-Based Methods: This category encompasses techniques wherein attributes are meticulously traced by domain experts.
• Deep Learning Approaches: This classification pertains to methods that employ deep learning algorithms for feature extraction and pattern recognition tasks [31].

Embedded vision systems are quintessentially integrated within a paradigm known as advanced driver assistance systems (ADASs), depicted in Figure 1 [32]. These systems harness sensorial input, predominantly from cameras, to assimilate environmental data. Subsequently, they synthesize decisions that align with both the perceived information and predefined objectives, thereby influencing the environment through iterative actions. This cyclical process persists until the attainment of the designated goal [33]. Specifically, within autonomous vehicular technology, the perception component is pivotal for obstacle detection including other vehicles, pedestrians, and miscellaneous objects, as well as for infrastructure recognition and self-localization within the milieu. It is customary to categorize perception systems into distinct classes based on their functional attributes and sensor modalities.

Road signs and road lanes play a vital role in allowing traffic to develop under very good conditions (speed, safety), but we would nevertheless like to remind you that

-

Sign supports must not encroach on the left and right lane and must be positioned as far as possible from surfaces accessible to vehicles.

-

Supports for gantries, jibs, etc., must generally be isolated by safety guardrails.

The role of road signs is to

• To make road traffic safer and easier.
• To remind you of certain traffic regulations.
• To indicate and remind the various special regulations.
• To provide information about the road user.

To structure this comprehensive review and guide the reader through its analytical goals, the following research questions are addressed:

• RQ1: What are the most efficient embedded vision techniques for road element detection?
• RQ2: What are the current limitations of these systems under real-world constraints?
• RQ3: Which unexplored areas can lead to safer, more scalable ADAS integration?
• RQ4: What types of processors are most commonly used in embedded vision systems for traffic sign and lane detection, and how do they impact system performance under real-time constraints?

The objective of this manuscript is to furnish a comprehensive survey delineating the myriad methodologies employed in the detection and recognition of traffic signs and lane markings within the realm of embedded vision systems. Section 2 delineates the categorization of signage into horizontal and vertical typologies. Section 3 proffers an exhaustive overview of the extant systems prevalent within the automotive sector, accompanied by a systematic critique. Section 4 scrutinizes the prevalent Image Processing (IM) and Artificial Intelligence (AI) [34] techniques utilized for the localization, detection, and recognition of traffic elements, analyzing the most ubiquitously adopted methodologies. Section 5 deliberates on extensive empirical experimentation pertaining to detection and recognition processes. Lastly, Section 6 encapsulates the findings and implications of the study.

2. Materials and Methods

To ensure clarity, focus, and scientific rigor of this review, our primary aim is to assist researchers working specifically on the development of embedded vision systems in the automotive sector, with a particular emphasis on road user safety. Rather than addressing a general audience, this paper is intended as a research-oriented contribution, offering both a structured overview of existing techniques and a critical discussion of ongoing challenges and future opportunities. To structure this narrative review, we followed the SALSA framework (Search, Appraisal, Synthesis, and Analysis) (

Table 1. Overview of the SALSA methodology applied in this review.

SALSA Step	Description
Search	A comprehensive literature search was conducted using major scientific databases, including IEEE Xplore, MDPI, SpringerLink, ScienceDirect, and Wiley, covering the period from 2010 to 2025. Boolean keyword combinations such as “traffic sign recognition”, “lane detection”, “ADAS”, “embedded vision systems”, and “deep learning in automotive” were employed. The search yielded over 200 potentially relevant sources, including journal articles, conference papers, and technical reports.
Appraisal	A systematic selection process was applied based on predefined inclusion and exclusion criteria. Selected studies demonstrated scientific rigor, empirical contributions, and direct relevance to embedded vision systems for road safety. Non-peer-reviewed materials, non-empirical works, and out-of-scope studies were excluded. The final pool of studies reflects the most impactful contributions to the field.
Synthesis	The retained articles were categorized into six thematic domains: (i) traffic sign detection and recognition, (ii) lane detection and departure warning, (iii) vision algorithms and deep learning models, (iv) sensor fusion and embedded architectures, (v) benchmark datasets and evaluation metrics, and (vi) hardware implementation and real-time constraints. This structure provides a coherent synthesis of the state-of-the-art technologies.
Analysis	The categorized works were critically analyzed to identify strengths, limitations, and future direction. Special focus was placed on real-time challenges, robustness under adverse conditions, hardware-software integration, and dataset limitations. This analysis helps highlight technological gaps and propose avenues for further research in intelligent embedded automotive systems.

). This methodology not only helps narrow the scope of relevant studies but also highlights technical advances, gaps, and implementation barriers within real-world embedded automotive systems.

2.1. Types of Signs

Several techniques or identification methods can be applied for the detection and recognition of road signs and road markings to extract and identify them in an urban environment, for example, using image processing techniques, computer vision, and artificial intelligence. An embedded vision system allows changing the information with a large number of devices on the road:

❖

Road infrastructure server;

❖

From one vehicle to another.

In the context of this inquiry, we commence with a succinct exposition on the typologies of road signs employed for the regulation of vehicular flow. The taxonomy of these signs is bifurcated into two categories: vertical signs, which are predominantly utilized for conveying regulations, warnings, and guidance in a perpendicular orientation to the road; and horizontal signs, which are marked directly on the road surface to delineate traffic lanes, convey directional information, and provide other operational instructions [34,35,36].

Road signs and markings are key elements for safe and structured traffic flow. Horizontal markings such as dashed, solid, dashed–solid, and solid–dashed lines define lane boundaries and dictate overtaking or lane-changing rules (Figure 2). Dashed lines allow movement between lanes, while solid or double solid lines indicate restrictions or prohibitions. Vertical road signs (Figure 3, Figure 4, Figure 5 and Figure 6) provide regulatory, warning, informational, and mandatory messages to drivers. These include circular prohibition signs, triangular warning signs, rectangular information signs, and round blue obligation signs, all of which follow standardized shapes and colors to support quick recognition [37,38,39].

Both horizontal and vertical signs are essential visual cues interpreted by embedded vision systems in modern vehicles. Their standardization enables accurate real-time detection, which is crucial for advanced driver assistance systems (ADASs) and autonomous driving functions. Recognizing these signals helps systems navigate roads, enforce safety rules, and interact effectively with the traffic environment. The upcoming section introduces commonly used optical character detection and recognition techniques in such embedded vision applications.

According to what was explained before, generally, a road sign can have a triangular, square, or rectangular shape. It should be noted that the style or shape of road signs is standardized. Based on these characteristics, we propose, in the subsequent section, the most commonly employed techniques in systems for optical character detection and recognition.

2.2. The Different Systems Existing in the Automotive Industry

Nowadays, there are many traffic sign recognition systems in the automotive sector. Currently, there are some car brands that have already been equipped with these TSR systems. Below is an example of a TSR application in a car (Figure 7).

a.

Advantages of TSR Systems

• Facilitate accurate traffic sign readings;
• Ensure interrupted performance;
• Provide complete speed solutions (avoid accidents);
• Ability to encrypt image data with other systems;
• Reads all types of infrared and non-reflective infrared plates.

b.

Limitations of TSR Systems

• A road sign outside the camera’s detection zone will not be detected.
• The systems operate only within the limits of the system and assist the driver.
• The driver must remain attentive while driving and remains fully responsible for their actions.

When it comes to TSR systems integrated in vehicles, there are two types of TSR technology applications:

• Passive application: consists of informing the driver by means of pictograms or sounds that they are entering a zone that has a new limit indicated by a traffic light. In this case, it will be the driver’s decision whether they obey the signal or not.
• Active application: consists of automatically intervening in the car when it detects a sign. For example, if the driver is driving at an excessive speed and the TSR system detects a stop sign but the car does not interpret that the driver intends to stop, the command to brake will be sent directly to the car to avoid a possible accident.

2.3. Horizontal Signs: Road Lanes

The utilization of image processing in the design and development of embedded vision systems for the detection and recognition of traffic signs and road markings is a critical aspect of modern vehicular technology. These systems integrate advanced algorithms that analyze visual data to identify and interpret traffic signs and lane information, thereby enhancing the safety and efficiency of navigation in both driver-assisted and autonomous vehicles [40]. The sophistication of these systems lies in their ability to process complex visual inputs with high accuracy and speed, contributing significantly to the advancement of intelligent transportation systems. Image processing techniques are fundamental to the development of embedded vision systems in the automotive sector [41]. These techniques enable vehicles to interpret and understand the environment around them, which is crucial for both driver assistance and autonomous driving applications. Here is a deeper look into their essential role:

• Enhanced Perception of Safety: Image processing algorithms are key to enhancing a vehicle’s perception of its surroundings. They process visual data from cameras to detect objects, lanes, signs, and pedestrians, which is vital for safety features like collision avoidance and lane-keeping assistance [34,42].
• Real-time processing: Embedded vision systems must process and analyze visual data in real time to be effective. Image processing techniques allow for the quick interpretation of data, enabling immediate responses to dynamic road conditions [43].
• Machine Learning Integration: The integration of machine learning with image processing has led to more accurate and adaptive vision systems. These systems can learn from vast amounts of data, improving their ability to recognize and respond to various traffic scenarios over time [44].

• Reduced Computational Load: Advanced image processing techniques help in reducing the computational load on embedded systems. By preprocessing visual data and extracting relevant features, these systems can operate efficiently without compromising on speed or accuracy.
• Sensor Fusion: Although camera-based vision systems are fundamental in enabling perception for autonomous vehicles, they exhibit several inherent limitations. Environmental conditions such as fog, shadows, glare, or heavy rain can reduce the reliability of image-based detection. Moreover, vision alone, especially from monocular cameras, fails to provide accurate depth information, which is critical for tasks like distance estimation and obstacle avoidance. In contrast, sensors like LiDAR and radar offer more consistent depth measurements and greater resilience to weather variations, though they may lack the resolution and semantic detail provided by visual sensors. By combining complementary sensor modalities, sensor fusion techniques significantly enhance perception reliability. As demonstrated in [45], fusing visual data with LiDAR or radar improves detection confidence, reduces false positives, and strengthens decision-making, particularly in edge-case scenarios [46].

In summary, image processing is a cornerstone of modern automotive technology, enabling vehicles to see and interpret the world with precision and intelligence. As the automotive industry continues to evolve towards autonomous driving, the role of image processing in embedded vision systems becomes increasingly significant.

2.4. Harnessing Artificial Intelligence in the Automotive Sector

Artificial intelligence (AI) refers to a range of technologies enabling the simulation of intelligent behavior and the automated execution of perception, comprehension, and decision-making tasks. These methodologies primarily encompass mathematics (particularly statistics), computer science, electronics, cognitive science, and neuroscience. Consequently, artificial intelligence has developed very strongly for more than 10 years, with an acceleration in the last 5 years, to allow uses such as [15,47,48]

• Visual perception: object recognition or scene description.
• Understanding of written or spoken natural language: automatic translation, automatic production of press articles, and sentiment analysis.
• Automatic analysis by “understanding” a query and returning relevant results, even if the result does not contain the words of the query.
• Autonomous decision-making for ADASs and autonomous vehicles.

AI currently requires considerable data and computational resources to learn efficiently. Research is now developing techniques to reduce energy consumption and limit the need for data [49] and other techniques to allow a solution to be generalized to multiple uses or to make AI robust to a single disruptive event.

a.

Exploring Artificial Intelligence Technologies

Even though artificial intelligence is mainly associated with mathematical disciplines and algorithmic techniques, it also includes other aspects to support comprehensive use, as shown in Figure 8. The main building blocks of an AI system are digital platforms (user data) or sensor infrastructure [45,50] (machine or environmental data) to generate regular data/events flow. This infrastructure consists of networks of sensors strategically deployed to capture and transmit data from the physical world to computational systems for analysis and interpretation [51,52,53]. A communication network allows the collection of data/events used by the AI. This data must be sufficiently representative of the use case we are trying to address. By taking these special considerations into account and implementing robust data collection strategies, organizations can ensure that the data captured by sensor infrastructure is sufficiently representative of the specific use case, enabling more accurate information to be obtained and informed decisions to be made [54].

A hyperscale computing infrastructure stores and exploits the data streams in a reasonable timeframe. Globally, hyperscale IT infrastructure [55] represents a paradigm shift in data center design and operation, enabling organizations to efficiently and cost-effectively support the growing demands of the digital enterprise, cloud computing, big data analytics, and emerging technologies such as AI/ML [56]. Artificial intelligence algorithmic technologies (machine learning, deep learning, neural networks, etc.), a performance measurement, an error measurement, and a set of reference events for learning are required. These performance measurement techniques are essential for assessing the effectiveness and reliability of AI algorithms in different tasks and domains. By carefully choosing appropriate metrics and methodologies, researchers and practitioners can gain valuable insights into algorithm performance, identify areas for improvement, and make informed decisions about algorithm selection and deployment [57].

A simple man/machine interface of the platform types accessible via mobile or computer for “decision support”-type uses or an advanced interface (drone, robot, autonomous vehicle) for “autonomous decision”-type uses is the element that associates an algorithmic technology with a sectorial use [58].

b.

The usefulness of AI for embedded vision systems in the automotive sector

The application of AI in embedded vision systems is pivotal for the progression towards fully autonomous vehicles, as it allows for the interpretation of complex visual data, ensuring safety and reliability of diverse driving conditions. Moreover, AI models and methods have been systematically reviewed for their application in automotive manufacturing, highlighting the potential and applicability of AI for automotive original equipment manufacturers (OEMs) and suppliers [47]. Furthermore, advancements in computer vision algorithms and remote sensing data fusion techniques contribute significantly to the automotive sector by enhancing visual perception and navigation systems [59]. In the realm of automotive technology, AI-driven embedded vision systems are pivotal for the evolution of vehicles that are not only autonomous but also cognizant of safety and operational efficiency. The burgeoning corpus of scholarly articles attests to the critical role and swift progression of AI in this domain. The recent literature has elucidated the integration of deep learning with autonomous vehicle systems (AVSs), significantly impacting road safety and the future of transportation [60].

These advancements are particularly noteworthy in the context of sensor fusion cost reduction and the amalgamation of top-tier solutions to address uncertainties on the road. The systematic review of AVS employing deep learning spans a decade, focusing on RGB (Red, Green, and Blue) camera vision and its application in various vehicular functions, from perception analysis to augmented reality-based head-up displays [61]. Furthermore, the digitalization of the automotive sector accelerates the convergence of perception, computing, connectivity, and data fusion for Electric, Connected, Autonomous, and Shared (ECAS) vehicles. This convergence fosters cutting-edge computing paradigms with embedded cognitive capabilities, propelling vehicles towards sustainable green mobility through intelligent functions and automotive digital infrastructure [62].

The challenges of embedding computer vision in automotive safety systems are manifold, yet the continuous innovation in AI methodologies and virtual validation and testing is paving the way for more advanced and reliable ECAS architectures [63]. As vehicles become increasingly intelligent and interconnected, functioning as edge microservers on wheels, the embedded AI facilitates a myriad of autonomous capabilities, enhancing the overall driving experience and safety [62]. In essence, AI for embedded vision systems is not merely a technological enhancement but a transformative force driving the automotive sector towards an era of intelligent mobility. The commitment to research and development in this field is crucial for realizing the full potential of AI in automotive applications, ensuring that the vehicles of tomorrow are equipped to navigate the complexities of real-world driving environments.

Furthermore, the following diagram (Figure 9) shows the evolution of processor sales by economic sector over the last few years. The industrial, tertiary (including financial services), and commercial sectors are the three sectors that consume the most chips. This number is growing overall in all sectors, apart from the economic crisis of 2008. GPU-based processors (led by NVIDIA) currently dominate the AI gas pedal market. Other technological approaches are still in the research and development stage. Just like the GPU, which took a decade to become established, solutions should be available by 2020/2025. They focus on solving specific AI problems, once the learning phase is over, such as the inference phase (FPGA) [57,58] or the use of constrained consumption (QUALCOMM) on mobile devices.

2.5. Vision Transformers in Object Detection and Tracking

Vision Transformers (ViTs) have recently emerged as strong alternatives to CNN-based (convolution neural network) approaches for object detection and tracking, particularly in autonomous driving scenarios. Unlike convolutional models, ViTs utilize self-attention mechanisms to capture the global context, which improves detection performance in complex visual scenes. However, their computational demands often limit their use in embedded systems. To address this, recent works have focused on designing lightweight and hybrid models. For example, MobileViT integrates convolutions with transformer blocks to reduce latency while maintaining accuracy, showing promising results in mobile and real-time platforms [64]. Similarly, DeViT introduces a modular design that partitions the model across low-power devices for efficient inference without significant accuracy degradation [65]. Fast-COS, introduced by [66], proposes reparametrized attention tailored for driving scenes, outperforming traditional models like YOLO and FCOS in terms of speed and throughput on edge devices. Moreover, Lai-Dang highlighted the importance of software–hardware co-design and model compression in deploying ViTs under strict resource constraints [67]. These developments underline a key trade-off: ViTs offer better detection accuracy, but only recent innovations have made their deployment feasible on embedded platforms. A comparative discussion of these models is essential to reflect the current state of the art and guide future research.

3. Related Work

This section is divided by subheadings. It should provide a concise and precise description of the experimental results, their interpretation, and the experimental conclusions that can be drawn.

The proposal of an embedded vision system should efficiently and accurately detect and recognize traffic signs, while also robustly identifying road edges. These systems play a vital role in functions such as driver assistance systems [68], pedestrian detection, lane departure warning, traffic sign recognition, and autonomous driving. By leveraging AI algorithms and advanced image processing techniques, these systems can accurately detect and recognize objects, pedestrians, road signs, and lane markings, providing invaluable insights to both drivers and autonomous driving systems. As automotive manufacturers continue to invest in the research and development of embedded vision technologies, we can expect further advancements that will drive the future of transportation towards safer, smarter, and more efficient vehicles.

Traffic signs are characterized by regions of interest that can identify them in road traffic (color and geometric shape), followed by the recognition or classification of the detected signs according to the type of their pictograms. For road edges, detection is based on a model of the road, followed by the recognition or estimation of the steering angle of wheels according to the scenario (left, middle, or right), in order to assist the driver during the conduction period.

A Multi-Cue Visual Tracking Framework refers to a system in computer vision designed to track objects using a combination of visual cues, which can originate from a single modality or multiple sources. Single-modal tracking relies solely on one type of input, typically visual data, while multimodal tracking integrates various inputs such as depth, motion, or thermal information to enhance robustness. These frameworks aim to improve accuracy and adaptability by incorporating features like color, shape, texture, and motion.

As illustrated in Figure 10, visual tracking methods are generally divided into two major categories: traditional tracking approaches and deep learning-based approaches. Traditional methods include stochastic, deterministic, generative, and discriminative strategies, which are typically based on handcrafted features and fixed models. In contrast, deep learning-based methods, including those that utilize deep or hyper features, offer more flexible and data-driven solutions that adapt to complex environments and object variations. This comparison highlights the evolution from rule-based models to learning-based systems, reflecting a broader trend in the field toward higher accuracy and scalability. Implementation of these frameworks involves algorithms for feature extraction, object representation, and model updating, often leveraging modern machine learning techniques. Ongoing research continues to explore more efficient and reliable tracking solutions, particularly for use in real-time, dynamic settings such as autonomous vehicles, surveillance, and interactive systems.

3.1. Traditional Tracking Approach

The traditional tracking approach in computer vision relies on predefined rules and manually designed features for object tracking in a video. It involves extracting features like color, texture, or edges, creating a model or template for object representation, and estimating motion using methods like optical flow or template matching. Matching and localization are performed across frames, often employing correlation-based techniques. Filtering methods, such as Kalman filters, are used for accuracy improvement [69].

Each tracking method offers distinct strengths (

Table 2. Two-dimensional tracking approaches and their mathematical models.

Tracking Approach	Description	Equations and Parameters
Stochastic Method	Incorporates randomness and probability to model uncertainty in data, used in noisy or dynamic scenarios [34,70,71].	Gaussian Distribution:
		$\mathrm{P}\left(\mathrm{x}\right)={\displaystyle \frac{1}{\sqrt{2\mathsf{\pi }{\mathsf{\sigma }}^2}}}{\mathrm{e}}^{-{\displaystyle \frac{(\mathrm{x}-\mathsf{\mu }{)}^2}{2{\mathsf{\sigma }}^2}}}$	(1)
		Kalman Filter:
		${\mathrm{x}}_{\mathrm{t}+1}=\mathrm{A}{\mathrm{x}}_{\mathrm{t}}+\mathrm{B}{\mathrm{u}}_{\mathrm{t}}+{\mathrm{w}}_{\mathrm{t}}$	(2)
		${\mathrm{z}}_{\mathrm{t}}=\mathrm{H}{\mathrm{x}}_{\mathrm{t}}+{\mathrm{v}}_{\mathrm{t}}$	(3)
		Monte Carlo:
		$\mathrm{I}\approx {\displaystyle \frac{1}{\mathrm{N}}}{\displaystyle \sum _{\mathrm{i}=1}^{\mathrm{N}}}\mathrm{f}\left({\mathrm{x}}_{\mathrm{i}}\right)$	(4)
		MCMC:
		$\mathrm{P}\left(\left.{\mathrm{x}}_{\mathrm{i}+1}\right|{\mathrm{x}}_{\mathrm{i}}\right)=\mathrm{T}\left(\left.{\mathrm{x}}_{\mathrm{i}+1}\right|{\mathrm{x}}_{\mathrm{i}}\right)\mathrm{P}\left({\mathrm{x}}_{\mathrm{t}}\right)$	(5)
		SGD:
		${\theta }_{\mathrm{t}+1}={\theta }_{\mathrm{t}}-\mathsf{\eta }.\nabla \mathrm{L}\left({\theta }_{\mathrm{t}},{\mathrm{x}}_{\mathrm{t}}\right)$	(6)
Deterministic Method	Uses predefined rules for consistent outputs, effective in stable conditions [72,73].	Image Filtering:
		${\mathrm{I}}^{\prime }\left(\mathrm{x},\mathrm{y}\right)={\displaystyle \sum _{\mathrm{i}=-\mathrm{m}}^{\mathrm{m}}}{\displaystyle \sum _{\mathrm{j}=-\mathrm{n}}^{\mathrm{n}}}\mathrm{K}\left(\mathrm{i},\mathrm{j}\right).\mathrm{I}(\mathrm{x}-\mathrm{i},\mathrm{y}-\mathrm{j})$	(7)
		Edge Detection:
		$\mathrm{G}\left(\mathrm{x},\mathrm{y}\right)=\sqrt{{\mathrm{G}}_{\mathrm{x}}^2\left(\mathrm{x},\mathrm{y}\right)+{\mathrm{G}}_{\mathrm{y}}^2\left(\mathrm{x},\mathrm{y}\right)}$	(8)
		Geometric Transform:
		P′ = T·P	(9)
Generative Method	Models the underlying data distribution P(X), useful for image synthesis and representation learning [74].	Data Distribution:
		$\mathrm{P}\left(\mathrm{X}\right)=\int \mathrm{P}\left(\mathrm{X}|\mathrm{Z}\right)\mathrm{P}\left(\mathrm{Z}\right)\mathrm{d}\mathrm{Z}$	(10)
		GAN Objective:
		${\mathrm{m}\mathrm{i}\mathrm{n}}_{\mathrm{G}}{\mathrm{m}\mathrm{a}\mathrm{x}}_{\mathrm{D}}{\mathsf{{\rm E}}}_{\mathrm{X}~{\mathrm{P}}_{\mathrm{d}\mathrm{a}\mathrm{t}\mathrm{a}}}\left[\mathrm{l}\mathrm{o}\mathrm{g}\mathrm{D}\left(\mathrm{X}\right)\right]+{\mathsf{{\rm E}}}_{\mathrm{X}~{\mathrm{P}}_{\mathrm{Z}}}\left[\mathrm{l}\mathrm{o}\mathrm{g}(1-\mathrm{D}\left(\mathrm{G}\left(\mathrm{Z}\right)\right))\right]$	(11)
		VAE Loss:
		$\mathrm{L}={\mathsf{{\rm E}}}_{\mathrm{Z}~\mathrm{q}\left(\mathrm{Z}|\mathrm{X}\right)}\left[\mathrm{l}\mathrm{o}\mathrm{g}\mathrm{P}\left(\mathrm{X}|\mathrm{Z}\right)\right]-\mathrm{K}\mathrm{L}\left(\mathrm{q}\left(\mathrm{Z}|\mathrm{X}\right)\Vert \mathrm{P}\left(\mathrm{Z}\right)\right)$	(12)
Discriminative Method	Learns decision boundaries between classes by modeling P(Y|X), applied in classification and detection [75,76].	Logistic Regression:
		$\mathrm{P}\left(\mathrm{Y}=1|\mathrm{X}\right)={\displaystyle \frac{1}{1+{\mathrm{e}}^{-({\mathsf{\beta }}_0+{\mathsf{\beta }}^{\mathrm{T}}\mathrm{X})}}}$	(13)
		SVM Margin:
		$\mathrm{M}={\displaystyle \frac{2}{‖\mathrm{w}‖}}$	(14)
		Neural Network Loss:
		$\mathrm{L}=-{\displaystyle \frac{1}{\mathrm{N}}}{\displaystyle \sum _{\mathrm{i}=1}^{\mathrm{N}}}\left[{\mathrm{y}}_{\mathrm{i}}\log \left(\widehat{{\mathrm{y}}_{\mathrm{i}}}\right)+\left(1-{\mathrm{y}}_{\mathrm{i}}\right)\log \left(1-\widehat{{\mathrm{y}}_{\mathrm{i}}}\right)\right]$	(15)

): deterministic approaches ensure stability and precision in structured tasks, while stochastic methods enhance adaptability under uncertainty. Generative models enable synthetic data creation by learning data distributions, supporting augmentation and representation learning.

The discriminative methods excel in classification by learning decision boundaries, but require labeling data. The choice of method depends on the task complexity, data variability, and system objectives.

3.2. Deep Leaming Based Tracking Approach

In tracking objects in video sequences, a deep learning-based approach employs neural networks, particularly deep ones, to enhance tracking performance. This methodology leverages convolutional neural networks (CNNs) for extracting hierarchical features, Siamese networks for discrimination between target objects and backgrounds, and recurrent neural networks (RNNs) or Long Short-Term Memory (LSTM) networks to capture temporal dependencies. Object detection networks like region-based CNNs (R-CNN) or You Only Look Once (YOLO) are integrated for simultaneous detection and tracking. Online learning and fine-tuning strategies allow adaptation during tracking. Tracking-by-detection methods use pre-trained object detectors, and data augmentation techniques enhance generalizations. Challenges include addressing occlusions, scale, and pose variations and maintaining real-time performance. Some approaches strive for end-to-end learning, directly processing raw input frames to object tracks without explicit feature engineering. These methods showcase improved accuracy, especially in scenarios with complex motions and appearance variations, evolving with advancements in neural network architectures and training strategies.

This

Table 3. Deep feature extraction methods in visual tracking.

Tracking Approach	Description	Equations and Parameters
Deep Feature-Based	Uses hierarchical CNN representations and transfer learning (e.g., VGG16, ResNet) for classification and detection tasks [77].	Weight update:
		${\mathrm{w}}_{\mathrm{t}+1}={\mathrm{w}}_{\mathrm{t}}-\mathsf{\eta }.\nabla \mathrm{L}\left({\mathrm{w}}_{\mathrm{i}}\right)$	(16)
		Feature map:
		$\mathrm{F}=\mathrm{f}\left(\mathrm{W}.\mathrm{X}+\mathrm{b}\right)$	(17)
		Softmax:
		$\mathrm{P}(\mathrm{y})=\left(\mathrm{c}|\mathrm{x}\right)={\displaystyle \frac{{\mathrm{e}}^{{\mathrm{z}}_{\mathrm{c}}}}{{\sum }_{\mathrm{i}=1}^{\mathrm{C}}{\mathrm{e}}^{{\mathrm{z}}_{\mathrm{i}}}}}$	(18)
Hyper Feature-Based	Combines handcrafted (HOG, color histograms) and CNN features using multi-modal integration for robust tracking [78,79].	HOG gradient:
		$\mathrm{H}\mathrm{O}\mathrm{G}={\displaystyle \sum _{(\mathrm{x},\mathrm{y})ϵ\mathrm{R}}}\left.\left|\nabla \mathrm{I}(\mathrm{x},\mathrm{y})\right.\right|$	(19)
		Unified graph fusion:
		${\mathrm{F}}_{\mathrm{c}\mathrm{o}\mathrm{m}\mathrm{b}\mathrm{i}\mathrm{n}\mathrm{e}\mathrm{d}}=\mathsf{\alpha }{\mathrm{F}}_{\mathrm{s}\mathrm{p}\mathrm{a}\mathrm{r}\mathrm{e}}+\left(1-\mathsf{\alpha }\right){\mathrm{F}}_{\mathrm{d}\mathrm{e}\mathrm{n}\mathrm{s}\mathrm{e}}$	(20)
Transformer-Based (ViT)	Uses self-attention mechanisms for global feature modeling; models like MobileViT and Fast-COS improve runtime performance [80].	Self-attention:
		$\mathrm{A}\left(\mathrm{Q},\mathrm{K},\mathrm{V}\right)={\displaystyle \frac{\mathrm{Q}{\mathrm{K}}^{\mathrm{T}}}{\sqrt{{\mathrm{d}}_{\mathrm{K}}}}}.\mathrm{V}$	(21)

summarizes advanced tracking strategies leveraging deep learning and hybrid features. Deep and hyper features improve robustness by combining semantic and handcrafted cues, while transformer-based models offer global attention mechanisms for superior context modeling. These methods significantly enhance accuracy and adaptability in complex visual environments.

3.3. Three-Dimensional Perception and Structured Scene Understanding

Current research in intelligent transportation systems increasingly leverages 3D perception and structured scene understanding to complement traditional 2D approaches. In railway applications, RailVoxelDet introduces a lightweight voxel-based LiDAR pipeline optimized for long-range object detection with competitive inference speed and computational efficiency [81,82]. Similarly, Rail-PillarNet builds upon PointPillars with a parallel attention encoder and transfer learning enhancements to detect foreign objects effectively without substantial parameter increase.

Beyond railway scenarios, Bird’s-Eye-View (BEV) representations have become a standard for structuring multi-sensor data. Frameworks like BEVFormer learn unified BEV features via spatio-temporal transformers, effectively fusing camera and LiDAR data into a robust 3D scene understanding model [83]. End-to-end fusion models such as BevDrive integrate cameras–LiDAR fusion directly into motion planning via BEV constructs, demonstrating the value of perception planning synergy [84].

Another emerging area is non-destructive measurement and interpretable diagnostics, especially relevant in safety-critical systems. For example, methods using physical constraints-guided quadratic neural networks enable interpretable fault diagnosis without needing fault samples, enhancing reliability through explainability, a trend gaining traction for embedding perception into maintenance workflows. To provide a concise overview of the mathematical foundations supporting recent 3D perception methods,

Table 4. Representative mathematical models in 3D perception and structured scene understanding.

3D Approach	Description	Equations and Parameters
Lightweight 3D Detection (Voxel-Based)	LiDAR point clouds are discretized into voxel grids, and features are aggregated per voxel to reduce computation [85].	Voxelization:
		$\mathrm{v}\left(\mathrm{i},\mathrm{j},\mathrm{k}\right)=\left\{\mathrm{p}=\left(\mathrm{x},\mathrm{y},\mathrm{z}\right)\mathrm{P}\right|\mathrm{x}ϵ{\mathrm{X}}_{\mathrm{i}},\mathrm{y}ϵ{\mathrm{X}}_{\mathrm{j}},\mathrm{z}ϵ{\mathrm{Z}}_{\mathrm{k}}\}$	(22)
		P = point cloud, ${\mathrm{X}}_{\mathrm{i}},{\mathrm{X}}_{\mathrm{j}},{\mathrm{Z}}_{\mathrm{k}}$ = voxel bin ranges.
Bird’s-Eye-View (BEV) Representation	Transforms 3D coordinates into a 2D top-down plane for detection/segmentation [83].	Projection:
		$\left(u,v\right)= \left\{{\displaystyle \frac{x}{z}} , {\displaystyle \frac{y}{z}}\right\}$	(23)
		(x, y, z) = 3D point, (u, v) = BEV coordinates.
Multi-Sensor Fusion	Features from LiDAR, camera, and radar are combined via weighted or attention-based fusion [85].	Weighted fusion:
		${\mathrm{F}}_{\mathrm{f}\mathrm{u}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}}=\mathsf{\alpha }{\mathrm{F}}_{\mathrm{L}\mathrm{i}\mathrm{D}\mathrm{A}\mathrm{R}}+\left(1-\mathsf{\alpha }\right){\mathrm{F}}_{\mathrm{C}\mathrm{a}\mathrm{m}\mathrm{e}\mathrm{r}\mathrm{a}}$	(24)
		${\mathrm{F}}_{\mathrm{L}\mathrm{i}\mathrm{D}\mathrm{A}\mathrm{R}}$, ${\mathrm{F}}_{\mathrm{C}\mathrm{a}\mathrm{m}\mathrm{e}\mathrm{r}\mathrm{a}}$ = feature vectors, α = fusion weight.
Non-Destructive Measurement (Quadratic NN)	Quadratic neural networks constrained by physics provide interpretable diagnostics under zero-fault conditions [86].	Quadratic model:
		$\mathrm{y}={\mathrm{x}}^{\mathrm{T}}{\mathrm{Q}}_{\mathrm{x}}+{\mathrm{W}}_{\mathrm{x}}+\mathrm{b}$	(25)
		x = input vector, Q = quadratic coefficient matrix, W = weight vector, b = bias.

summarizes representative approaches, their core equations, and key references.

Overall, these approaches reinforce the role of 3D perception as a necessary complement to traditional 2D methods, providing a more comprehensive perspective for intelligent transportation systems.

3.4. Dataset Description

Both traditional tracking methods and deep learning-based approaches fundamentally rely on comprehensive datasets of images and lane markings to achieve reliable performance. Traditional techniques, which utilize handcrafted features like edges and colors, require diverse image datasets to handle variations in lighting, weather, and road conditions. Likewise, deep learning methods demand well-annotated datasets for training and testing, allowing the models to learn and generalize effectively across different scenarios. To ensure robust real-world applications, datasets must encompass a wide range of environments, including challenging conditions like occlusions or extreme weather. Without such data, neither strategy can perform reliably or adapt to dynamic situations. These datasets cater to various aspects of the automotive sector, including lane detection, geolocation, traffic sign recognition, and driver behavior analysis, making them vital for advancing intelligent transportation systems.

3.4.1. German Traffic Sign Recognition Benchmark (GTSRB)

The GTSRB dataset is essential for training and evaluating automatic traffic sign classification algorithms, particularly for advanced driver assistance systems (ADASs) and autonomous vehicles [87]. GTSRB contains 43 classes of German traffic signs, with over 50,000 annotated images. The images vary significantly in terms of perspective, lighting, resolution, and environmental conditions (e.g., blur, shadows, and distortions). These characteristics make the dataset ideal for testing the robustness of models in real-world scenarios.

3.4.2. LISA Traffic Sign Dataset

This dataset is used to train models capable of recognizing American traffic signs, even in complex environments with variations in perspective, occlusions, and lighting conditions.

The LISA datatest contains over 6600 images with 7855 annotations covering 47 classes of signs [88]. The images are captured in urban and suburban environments. Each sign is annotated with detailed information about its position, size, and type, making it a valuable resource for training and testing visual recognition systems.

3.4.3. Cityscapes Dataset

Cityscapes is widely used for semantic segmentation of urban scenes, a crucial task for autonomous navigation and understanding complex road environments [89]. This dataset features thirty annotated classes, including eight related to traffic signs and road markings. It offers 5000 finely annotated images and 20,000 images with simplified annotations, captured in 50 European cities. The annotations include pedestrians, vehicles, signs, and other urban elements, enabling detailed interpretation of traffic scenarios.

3.4.4. TuSimple Lane Detection Dataset

This dataset is specifically designed for lane detection tasks, which are essential for lane-keeping systems and autonomous driving applications.

TuSimple provides 6408 annotated road images, primarily of highways. The annotations include precise lane line positions captured under various conditions (e.g., daylight and twilight). This dataset presents realistic challenges for developing reliable lane detection algorithms [90].

3.4.5. CULane Dataset

CULane is suited for studying road markings in complex urban environments, such as intersections and poorly marked roads. With approximately 133,000 annotated images, this dataset covers diverse scenarios, including roads bordered by vegetation, intersections, and unmarked areas [91]. The detailed annotations help identify both straight and curved lanes, making it ideal for navigation studies in real-world environments.

3.4.6. U.S. Traffic Signs Dataset

This dataset is critical for recognizing traffic signs specific to the United States, offering a wide range of scenarios to test the robustness of recognition models [92]. It includes several thousand annotated images representing over 100 classes of traffic signs. The images show variations in lighting, perspective, and weather conditions, reflecting real-life situations.

3.4.7. Traffic Sign Dataset—Classification

This dataset enables the study of traffic signs in European environments, providing data tailored to regional diversity of robust models. It contains thousands of annotated images across 58 classes of signs [93]. The images capture varied environments, from rural areas to large cities, under different weather conditions.

3.4.8. Caltech Pedestrian Dataset

This dataset is crucial for pedestrian detection, a fundamental task for ensuring the safety of autonomous driving systems in urban environments.

The dataset contains approximately 250,000 annotated images capturing pedestrians under diverse conditions, including varying lighting and crowd density [94]. These annotations help train models to detect pedestrians accurately and efficiently.

3.4.9. KITTI Dataset

The KITTI dataset is widely recognized in the autonomous driving community as a foundational benchmark for evaluating both visual and geometric perception algorithms. Collected using a real-world vehicle equipped with stereo cameras, a rotating LiDAR scanner, GPS, and inertial sensors, KITTI captures diverse urban and highway scenes with high precision. It includes ground truth annotations for object detection, lane estimation, optical flow, and depth prediction, enabling robust cross-modal training and validation of perception systems [95]. Its multimodal structure makes it particularly valuable for the development of end-to-end models that integrate both spatial and semantic understanding. Recent advancements in this area include Vision–Language Models (VLM) and Vision–Language–Action (VLA) frameworks, which unify perception and decision-making by leveraging aligned visual and textual data. These architectures enhance interpretability and improve decision robustness in complex urban scenarios [96].

3.4.10. Malaysia Roads Dataset

This dataset is designed for recognizing road markings and signs specific to Malaysian roads. It contains thousands of annotated images featuring road markings and signs unique to the region. This data is crucial for adapting visual recognition systems to local requirements [97].

3.4.11. STS Dataset (Simulated Traffic Sign Dataset)

The STS dataset is designed for training and testing algorithms for traffic sign recognition in simulated and real-world environments. This dataset includes a large number of traffic sign images generated through simulation tools to represent diverse conditions [98]. It focuses on various weather conditions, lighting scenarios, and perspectives. STS datasets are particularly useful for augmenting training datasets where real-world data is limited.

3.4.12. Belgian Traffic Sign Classification Dataset

The BTSC dataset is used for traffic sign classification tasks, specifically focusing on Belgian traffic signs. It supports the development of systems that recognize diverse regional traffic signs [99]. The dataset contains more than 10,000 images of 62 classes of Belgian traffic signs, including regulatory, warning, and informational signs. The images cover varying conditions such as partial occlusions, different angles, and lighting variations, making it highly suitable for robust model training.

3.4.13. Driver Inattention and Traffic Sign Dataset

The DITS dataset focuses on driver behavior analysis in relation to traffic sign recognition. It is useful for evaluating driver attention and alert systems. DITS includes synchronized data of driver behavior, traffic signs, and vehicle performance [100]. The dataset contains recordings of traffic sign encounters in various driving conditions, including challenging scenarios like night driving or adverse weather. It helps in developing systems that detect driver inattention and generate warnings when signs are ignored.

3.4.14. Text-Based Traffic Sign Dataset in Chinese and English (TTSDCE) Dataset

The Traffic Guide Panel dataset includes thousands of images focused on highway guide panels with English text, serving as a benchmark for text-based traffic sign recognition. Due to limited availability of annotated training data, the method relies on a self-collected dataset named TTSDCE. This dataset contains 1800 images featuring traffic signs in Chinese and English, collected from various sources such as cameras and online images. It demonstrates flexibility in multilingual scenarios and is expected to be released for research in the future.

3.4.15. Comprehensive Analysis and Comparison of Automotive Datasets

Table 5. Comparison of automotive datatests.

Dataset Name	Category	Number of Images	Number of Classes	Training Images	Validation Images	Test Images	Image Resolution	Annotation Type	Captured Environments	Data Augmentation	Metadata Availability	License	Official Link
GTSRB Dataset	Traffic Sign Recognition	50,000+	43	39,209	12,630	12,630	Varies	Bounding Box	Urban	Yes	No	Open Source	https://benchmark.ini.rub.de/ (accessed on 12 June 2025)
LISA Dataset	Traffic Sign Recognition	6610+	47	4700+	1000+	1000+	1280 × 960	Bounding Box	Urban/ Suburban	No	No	Open Source	https://cvrr.ucsd.edu/home (accessed on 12 June 2025)
Cityscapes Dataset	Urban Scene Segmentation	25,000+	30	5000	500	19,500	2048 × 1024	Pixel-Level	Urban	No	No	Open Source	https://www.cityscapes-dataset.com/ (accessed on 12 June 2025)
TuSimple Dataset	Lane Detection	6408	Not Applicable	3626	358	2782	1280 × 720	Lane Points	Highways	No	No	Open Source	https://paperswithcode.com/dataset/tusimple (accessed on 12 June 2025)
CULane Dataset	Lane Detection	133,000	Not Applicable	Approx. 100,000	Approx. 20,000	Approx. 13,000	1920 × 1080	Lane Points	Urban/Suburban	No	No	Open Source	https://xingangpan.github.io/projects/CULane.html (accessed on 12 June 2025)
U.S. Traffic Signs Dataset	Traffic Sign Recognition	Varies	100+	Varies	Varies	Varies	Varies	Bounding Box	Urban/Suburban	No	No	Proprietary	Not Publicly Available
Traffic Sign Dataset - Classification	Traffic Sign Recognition	6960	58	Varies	Varies	2000 files	Varies	Bounding Box	Urban/Rural	No	No	Open Source	https://www.kaggle.com/datasets/ahemateja19bec1025/traffic-sign-dataset-classification (accessed on 12 June 2025)
Caltech Pedestrian Dataset	Pedestrian Detection	250,000+	1	200,000+	30,000+	20,000+	640 × 480	Bounding Box	Urban	No	No	Open Source	https://www.kaggle.com/datasets/kalvinquackenbush/caltechpedestriandataset (accessed on 12 June 2025)
KITTI Dataset	Multiple Vision Tasks	Varies	Varies	Varies	Varies	Varies	Varies	Bounding Box, 3D	Mixed	Yes	Yes	Open Source	http://www.cvlibs.net/datasets/kitti/ (accessed on 12 June 2025)
Malaysia Roads Dataset	Road Markings	Thousands	Not Applicable	Varies	Varies	Varies	Varies	Bounding Box	Urban	No	No	Unknown	Not Publicly Available
GNSS Dataset	Geolocation	Varies	Not Applicable	Varies	Varies	Varies	Varies	Metadata	Mixed	No	Yes	Proprietary	Not Publicly Available
STS Dataset	Traffic Sign Recognition	Varies	Varies	Varies	Varies	Varies	Varies	Bounding Box	Mixed	No	No	Unknown	Not Publicly Available
BTSC Dataset	Traffic Sign Classification	10,000+	62	7000+	1500+	1500+	Varies	Bounding Box	Urban/Suburban	No	No	Open Source	https://btsd.ethz.ch/shareddata/ (accessed on 12 June 2025)
DITS Dataset	Driver Attention	Varies	Not Applicable	Varies	Varies	Varies	Varies	Driver Metadata	Mixed	No	Yes	Open Source	https://universe.roboflow.com/basharsss1998-gmail-com/dits (accessed on 12 June 2025)
TTSDCE Dataset	Traffic Signs	1800	Multi-ling	1500	N/A	300	300 × 300 to 1280 × 720	Bounding boxes, classes	Highways, urban streets	No	No	Open Source	http://www.aaroads.com (accessed on 12 June 2025)

offers an in-depth comparison of various automotive datasets, each tailored to specific research and application needs in the field of intelligent transportation systems. The datasets vary widely in terms of category, image volume, class diversity, and annotation types, reflecting their unique purposes. For instance, the GTSRB dataset and LISA datatest are focused on traffic sign recognition, boasting 43 and 47 classes, respectively, making them ideal for developing models that classify diverse road signs. In contrast, datasets like CULane and TuSimple prioritize lane detection tasks, with thousands of annotated images capturing road lane points under challenging conditions such as occlusions and poor lighting. The Cityscapes datatest stands out for its pixel-level annotations across 30 classes, making it a prime choice for urban scene segmentation in autonomous navigation.

The KITTI datatest and Caltech Pedestrian datatest diversify the landscape by targeting multiple vision tasks, including 3D bounding box annotations and pedestrian detection, respectively. While some datasets, such as the GNSS datatest, offer geospatial metadata crucial for localization, others like the STS datatest and BTSC datatest excel in providing robust traffic sign classification data. Additionally, datasets such as Malaysia Roads and Tunisian Road Signs are particularly valuable for region-specific research, ensuring adaptability across varying environments. The inclusion of details like resolution, annotation type, and metadata availability further highlights the datasets’ tailored applications. Datasets offering augmentation options, such as GTSRB, present an edge by simulating real-world complexities, whereas proprietary datasets like U.S. Traffic Signs and DITS underline the importance of accessible data-sharing initiatives in advancing the field. The Traffic Guide Panel dataset and the TTSDCE dataset are essential for advancing text-based traffic sign detection and recognition, particularly in multilingual contexts.

Each dataset, while addressing unique challenges, collectively forms a comprehensive foundation for developing innovative solutions in intelligent transportation, showcasing a balance between global applicability and specialized focus areas. This diversity ensures that researchers can choose the most appropriate datasets for their specific needs, whether the task involves recognizing a multitude of traffic signs, detecting lanes in varied conditions, or analyzing driver behavior in response to environmental cues.

Beyond the technical details summarized in Table 5, the comparison should also account for differences in licensing and commercial restrictions, as well as the strategies and costs of adapting models trained on synthetic data to real-world driving scenarios.

4. Results and Discussions

Identifying and recognizing traffic signs as well as road lane markings are essential for modern intelligent transportation systems and autonomous vehicle operations. Various innovative techniques rooted in artificial intelligence (AI) and image processing have been developed to enhance precision and robustness in these tasks. Deep learning methods, including convolutional neural networks (CNNs), are commonly utilized for classifying traffic signs, while object detection frameworks such as YOLO (You Only Look Once) and the Faster R-CNN have shown success in detecting signs in real time. For lane marking detection, segmentation models such as U-Net and SegNet are frequently applied, providing accurate delineation of road boundaries. Traditional approaches like edge detection using Canny and Sobel filters, alongside Hough Transform techniques, also play a significant role in identifying lane structures. Recent studies have further incorporated temporal analysis using neural networks like RNNs and LSTM to address dynamic road environments. To optimize performance, preprocessing techniques such as noise reduction, contrast adjustment, and region-of-interest (ROI) selection are often implemented.

While vision-based techniques offer detailed spatial and semantic information, they remain sensitive to variations in lighting, weather, and occlusion. Visual sensors alone cannot guarantee consistent performance across all conditions. As a result, many systems now incorporate data from complementary sensors, such as LiDAR and radar, which provide depth and robustness in challenging environments. This fusion of modalities helps compensate for the weaknesses of camera-only perception, making the overall system more reliable and better suited for real-world deployment. While the reviewed approaches highlight the strengths and weaknesses of 2D vision-based detection, several recent studies stress that extending perception into 3D provides a way to mitigate these weaknesses. Depth-aware cues from LiDAR or stereo enhance stability under occlusions, night driving, or adverse weather, where 2D methods alone often fail [101]. This perspective confirms that 3D is not a substitute but a complementary extension, ensuring more reliable intelligent transportation systems [102].

4.1. Image Processing Methods

Detection approaches vary depending on the methodology or strategy adopted, but they generally fall into three main categories. Firstly, there is the detection of object geometry. Secondly, there is color-based detection for localizing objects through segmentation, which identifies regions of interest. Lastly, artificial intelligence, particularly deep learning techniques, is increasingly employed for object detection tasks.

4.2. The Color-Based Methods

Color-based segmentation stands out as a swift detection method. Utilizing specific filters, algorithms swiftly extract regions of interest, enabling the identification of distinct objects. Nonetheless, fluctuations in lighting conditions can introduce inaccuracies or unwanted outcomes. To mitigate this challenge, researchers have explored diverse segmentation techniques based on various color spaces. These approaches find utilities across a spectrum of applications, including lane recognition, traffic sign detection, and vehicle license plate recognition.

4.2.1. The Methods That Use the RGB Space

The majority of algorithms directly use the color space of vision, i.e., RGB, provided that the parameters are fixed. Ruta [20] applied a segmented color enhancement technique, specifically using the colors yellow, blue, and red, with the choice of the channel of the dominant color in relation to the others made according to the following transformation (Equation (26)):

$$\begin{array}{c}{\mathrm{f}}_{\mathrm{R}}\left(\mathrm{x}\right)= \max (0,\min (\mathrm{r}-\mathrm{g},\mathrm{r}-\mathrm{b})/\mathrm{S}) \\ {\mathrm{f}}_{\mathrm{B}}\left(\mathrm{x}\right)= \max (0,\min (\mathrm{b}-\mathrm{r},\mathrm{b}-\mathrm{g})/\mathrm{S}) \\ {\mathrm{f}}_{\mathrm{Y}}\left(\mathrm{x}\right)= \max (0,\min (\mathrm{r}-\mathrm{b},\mathrm{g}-\mathrm{b})/\mathrm{S}) \end{array}$$

(26)

For each pixel, S = r + b + g; x = [r, g, b].

One downside of employing RGB space is its vulnerability to variations in lighting conditions, as we can encounter scenes with insufficient or excessive light that can cause problems in segmentation [103].

An approach based on SVFs (Simple Vector Filters) [104] (Formula (27)) uses f(r, g, b) < 1 for achromatic color and f(r, g, b) ≥ 1 for chromatic color. R, g, and b represent the brightness of each color, respectively, and D is the degree of extraction.

$$\mathrm{f}(\mathrm{r},\mathrm{g},\mathrm{b})={\displaystyle \frac{(\left|\mathrm{r}-\mathrm{g}\right|+\left|\mathrm{g}-\mathrm{b}\right|+|\mathrm{b}-\mathrm{r}|)}{3\mathrm{D}}}$$

(27)

The authors obtained good results in the separation of colors: red, blue, and yellow. For each pixel, x = [x_r, x_g, x_b], the SVF will be calculated according to the following Equation (28):

$$\begin{array}{c}\mathrm{Red} : \mathrm{r}-\mathrm{g} > 40 \mathrm{e} \mathrm{t} \mathrm{r}-\mathrm{b} > 40\\ \mathrm{Yellow} : \mathrm{r}-\mathrm{b} > 50 \mathrm{e} \mathrm{t} \mathrm{g} > 50\\ \mathrm{Blue} : \mathrm{b}-\mathrm{r} > 65 \mathrm{e} \mathrm{t} \mathrm{b} > 45\end{array}$$

(28)

The images are divided according to illumination changes into two classes: cloudy and sunny. The segmentation step takes 60 ms on average.

4.2.2. Methods That Use Non-Linear Color Spaces

Some algorithms use color enhancement techniques, which require color coordinate transforms, so Nnolim [8] applied color segmentation using non-linear color spaces. Studies using this solution have shown that the system is feasible. This includes the HSI and HSV color spaces. Equation (29) represents RGB transformation into HSV, as follows [105]:

$$\begin{array}{c}{\displaystyle \genfrac{}{}{0pt}{}{\mathrm{H}=\left\{\begin{array}{c}\frac{\mathrm{G} - \mathrm{B}}{\mathrm{S}} ; \mathrm{R}=\mathrm{V}\\ 2+\frac{\mathrm{B} - \mathrm{R}}{\mathrm{S}} ;\mathrm{G}=\mathrm{V}\\ 4+\frac{\mathrm{R} - \mathrm{G}}{\mathrm{S}} ;\mathrm{B}=\mathrm{V}\end{array}\right.}{\begin{array}{c}\mathrm{V}=\max \left(\mathrm{R},\mathrm{G},\mathrm{B}\right)\\ \mathrm{S}=\mathrm{V}-\min \left(\mathrm{R},\mathrm{G},\mathrm{B}\right)\end{array}}}\end{array}$$

(29)

In the HSV color space, hue represents the angle of rotation between the red plane and the origin, while saturation denotes the distance from any point in the color space to the color’s surface. The value (V) signifies the intensity or brightness attribute of the image’s channel.

Equations (30) and (31) for the conversion from RGB to HSI are given as follows:

$$\begin{array}{c}\mathrm{I}={\displaystyle \frac{\mathrm{R}+\mathrm{G}+\mathrm{B}}{3}}\\ \mathrm{S}=1-{\displaystyle \frac{\mathrm{m}\mathrm{i}\mathrm{n}(\mathrm{R},\mathrm{G},\mathrm{B})}{\mathrm{I}}}\\ \mathrm{H}=\left\{\begin{array}{c}\theta \mathrm{s}\mathrm{i} \mathrm{B}\le \mathrm{G}\\ 360-\theta \mathrm{B}>\mathrm{G}\end{array}\right.\end{array}$$

(30)

such that

$$\theta ={\cos }^{-1}({\displaystyle \frac{0.5\left[\left(\mathrm{R}-\mathrm{G}\right)+\left(\mathrm{R}-\mathrm{B}\right)\right]}{\sqrt{(\mathrm{R}-{\mathrm{G})}^2+(\mathrm{R}-\mathrm{B}\left)\right(\mathrm{G}-\mathrm{B})}}})$$

(31)

The HSI color space is grounded in human visual perception and color interpretation, characterized by a color hexagon, triangle, or circle model. Its attributes—hue, saturation, and intensity—bear resemblance to the HSV system, yet their mathematical formulations differ. In the hexagonal model, primary and secondary colors serve as vertices, while in the color triangle model, they are represented by line segments. The color wheel illustration demonstrates that primary and secondary colors are spaced apart by 120°.

Other authors [106] have used Otsu’s method with fuzzy logic in order to locate the regions of interest, applied exactly to extract the thresholds of the constituent elements (H, S, and V) of the HSV space, but the constraint that was posed is that the calculated distance between the colors is very close, because the HSV space is not symmetric. The method of Otsu applies automatic thresholding, compared to others that prefer to use manual thresholding [107], due to variations in the illumination of the RGB space. To limit the values of the optimal thresholds (Red, Blue) as is illustrated in

Table 6. The range of thresholds for the H, S, and V components of traffic signs.

Color	H	S	V
Blue	0.47 ≤ H ≤ 0.72||(0.85 ≤ H ≤ 1)	S ≥ 0.3	V ≥ 0.11
Red	0.03 ≤ H ≤ 0.94	S ≥ 0.15	V ≥ 0.07

, the pixel is considered red or blue if the H, S and V components meet the conditions. The advantage of this method is to eliminate noise.

Liu [108] also used manual thresholding, which performed well across different lighting conditions. Additionally, he introduced a novel thresholding approach that leverages color information from neighboring pixels to obtain output masks for each color using hue/saturation thresholding (Formula (32)).

$$\begin{array}{c}\mathrm{Red} (\mathrm{i},\mathrm{j})=\left\{\begin{array}{c}\mathrm{Vrai}, \mathrm{si} \mathrm{H}\left(\mathrm{i},\mathrm{j}\right)\le {\mathrm{T}\mathrm{h}\mathrm{R}}_1 \mathrm{ou} \mathrm{H}\left(\mathrm{i},\mathrm{j}\right)\ge {\mathrm{T}\mathrm{h}\mathrm{R}}_2\\ \mathrm{False}, \mathrm{otherwise}\end{array}\right.\\ \mathrm{Blue} (\mathrm{i},\mathrm{j})=\left\{\begin{array}{c}\mathrm{Vrai}, \mathrm{si} \mathrm{H}\left(\mathrm{i},\mathrm{j}\right)\le {\mathrm{T}\mathrm{h}\mathrm{B}}_1 \mathrm{et} \mathrm{H}\left(\mathrm{i},\mathrm{j}\right)\ge {\mathrm{T}\mathrm{h}\mathrm{B}}_2\\ \mathrm{False}, \mathrm{otherwise}\end{array}\right.\\ \mathrm{Yellow} (\mathrm{i},\mathrm{j})=\left\{\begin{array}{c}\mathrm{Vrai}, \mathrm{si} \mathrm{H}\left(\mathrm{i},\mathrm{j}\right)\le {\mathrm{T}\mathrm{h}\mathrm{Y}}_1\mathrm{et} \mathrm{H}\left(\mathrm{i},\mathrm{j}\right)\le {\mathrm{T}\mathrm{h}\mathrm{Y}}_2\\ \mathrm{et} \mathrm{S}\left(\mathrm{i},\mathrm{j}\right)\le {\mathrm{T}\mathrm{h}\mathrm{Y}}_3\\ \mathrm{False}, \mathrm{otherwise}\end{array}\right.\end{array}$$

(32)

where H and S are the hue and saturation channels; and ${ThY}_i$ $,{ThR}_i$ et ${ThB}_i$ are the fixed thresholds that can be found in [109].

The HSI space has two color components, hue and saturation, which are closely related to human perception. Hue represents the dominant value of color. Saturation represents the purity of color. HSI components can be obtained from RGB [110]. Moreover, this space is unstable near achromatic colors and cannot be used directly on these pixels. So the authors applied the HST technique [109] for the colors red, yellow, and blue, as shown in

Table 7. The range of thresholds for the HST space.

Couleur	ThR1	ThR2	ThB1	ThB2	ThY1	ThY2	ThY3
HST	10	300	190	270	20	60	150

. The obtained hue site H was in the range [0, 360], and the saturation S was between [0 and 255]. They also used the HEST method, based on the LUT technique, described by four parameters, h_min, h_top, h_max and s_min, shown in Figure 11, to improve the hue and saturation of colors that can be found in their paper [111]. It is in fact a flexible threshold, where different values are assigned using linear functions.

4.2.3. The Methods That Use Color Spaces Linear

Specific lighting conditions profoundly affect the color perception of road signs. Most approaches are based on HSV or HSI color space, but the camera used for this application has a raw Bayer output, and a conversion would be too computationally expensive due to the introduced nonlinearity. L. Song [112] chose to use the color space YUV, where Y represents the intensity, and U/V represents the color difference. The linear transformation between RGB and YUV is given by the following Formula (33):

$$\begin{array}{c}\left[\begin{array}{c}Y\\ U\\ V\end{array}\right]=\left[\begin{array}{ccc}0.299& 0.587& 0.114\\ -0.148& -0.289& 0.437\\ 0.615& -0.515& -0.100\end{array}\right]\left[\begin{array}{c}R\\ G\\ B\end{array}\right]\end{array}$$

(33)

H. T. Manjunatha [113] used the YCbCr color space. The proposed strategy converts the traffic signs captured with RGB space to YCbCr space. The color thresholding technique is utilized to isolate the red color, which represents hazard signs, and subsequently eliminate any extraneous elements from the images. The transformation (Equation (34)) from RGB to YCbCr is shown below [114]:

$$\begin{array}{c}\left[\begin{array}{c}Y\\ C_b\\ C_r\end{array}\right]=\left[\begin{array}{ccc}0.299& 0.587& 0.114\\ 0.596& -0.275& -0.321\\ 0.212& -0.523& -0.331\end{array}\right]\left[\begin{array}{c}R\\ G\\ B\end{array}\right]\end{array}$$

(34)

Implementing the different methods led us to find the best method (color space), according to [115]:

• In individual images, optimal outcomes were achieved using the RGB method; however, in videos, this was not the case.
• The HSV space gives higher results, but on condition that the execution time constraint is eliminated.

4.3. Geometry-Based Methods

Detecting road signs through color segmentation faces numerous challenges, including adverse weather conditions like snow or rain, varying times of the day (morning, night), and fluctuations in object distance between the camera and the road sign. These factors significantly impact the signs’ appearance due to light reflections and other environmental factors. As a result, approaches relying solely on color-based methods may lack effectiveness and robustness in perceiving the environment accurately. To address this, some researchers are exploring more reliable detection techniques focusing on the signs’ geometry. This involves analyzing the contours of the objects in the image to detect signs, thus minimizing overlap with surrounding objects in the road environment.

4.3.1. Hough Transform

The Detection of the Roadway

During forward motion, a vehicle can be considered to undergo continuous movement. When searching for lane lines within a specific area, it is essential to streamline image processing. In cases where lane lines are typically distributed on both the left and right sides of the road, the Hough transform proves to be a valuable technique for feature extraction [116]. This method detects objects with distinct shapes, such as straight lines, circles, and ellipses. X. Wei [117] presented a road line detection approach primarily based on Hough line detection statistics. The core idea involves converting the Cartesian coordinate system of the image into a polar coordinate Hough space, transforming (Equation (35)) each pixel coordinate (P (x, y)) into (ρ, θ).

The point P (x, y) on the same line is satisfied:

$$\begin{array}{c}{\displaystyle \genfrac{}{}{0pt}{}{\mathrm{x}.\cos \theta +\mathrm{y}.\sin \theta =\mathsf{\rho }}{\mathsf{\rho }=\sqrt{{\mathrm{S}}_{\mathrm{x}}^2+{\mathrm{S}}_{\mathrm{y}}^2}\mathrm{e}\mathrm{t}\theta ={\tan }^{-1}\frac{{\mathrm{S}}_{\mathrm{y}}}{{\mathrm{S}}_{\mathrm{x}}}}} \end{array}$$

(35)

$\rho$ t (S_x, S_y), and $\theta$ is the direction.

The discretization of $\rho$ and θ directly affects detection: coarse steps risk missing fine lane markings, while very fine steps increase computation and noise sensitivity. Selecting an adequate resolution is therefore a compromise between accuracy and real-time performance.

When utilizing an adjusted camera for lane line detection, there exists a substantial likelihood of the lane lines being obscured. Typically, these lines are situated on both the left and right sides of the roadway. To enhance the precision of the detection process, it is imperative to employ tracking technology, which augments both the velocity and accuracy of detection. This necessitates the application of the Hough transform to confine the scope of its voting space, as cited in [117]. Specifically, the parameters ρ and θ are adjusted to modulate the voting space’s range. The polar angle and polar radius are delimited by the boundary lines flanking the left and right sides, respectively.

The paramount criteria for traffic lane detection and tracking methods encompass both its velocity and dependability. W. Farage [118] introduced a novel approach termed LaneRTD (Lane Real-Time Detection), which leverages raw RGB imagery to delineate road margins, as depicted in Figure 12, through the segmentation of lane lines. The principal innovation of this method lies in its meticulous equilibrium between rapid processing, minimal resource consumption (notably in terms of memory and CPU usage), consistent reliability, and robustness, thereby satisfying the operational demands of systems designed for advanced driver assistance (ADAS) or autonomous vehicles. Predicated on the Hough Transform technique, this methodology incorporates a smoothing phase to mitigate the impact of discernible noise on the edge detection process, achieved through the application of a Gaussian filter. The Formula (36) for a Gaussian filter kernel with dimensions (2k + 1) × (2k + 1) is as follows:

$$\begin{array}{c}{\mathrm{H}}_{\mathrm{i}\mathrm{j}}={\displaystyle \frac{1}{2\mathsf{\pi }{\mathsf{\sigma }}^2}} {\mathrm{e}}^{\left(-{\displaystyle \frac{(\mathrm{i}-{\left(\mathrm{k}+1\right))}^2+(\mathrm{j}-{\left(\mathrm{k}+1\right))}^2}{2{\mathsf{\sigma }}^2}}\right)}\\ 1\le \mathrm{i},\mathrm{j}\le (2\mathrm{k}+1)\end{array}$$

(36)

The smoothing strength depends on σ and the kernel size k: larger values reduce noise but may blur lane edges, while smaller values preserve details but risk leaving residual noise. Proper tuning ensures a balance between robustness and efficiency.

For the purpose of edge detection and extraction, the Canny operator is employed. Given that an edge within an image may be oriented in various directions, the Canny method utilizes four distinct filters to accurately detect horizontal, vertical, and diagonal edges within the blurred image. The edge detection operator yields a value for the first derivative of both the horizontal direction (G_x) and the vertical direction (G_y). The gradient and direction of the edge can be ascertained as follows (Formula (37)):

$$\begin{array}{c}{\displaystyle \genfrac{}{}{0pt}{}{\mathrm{G}=\sqrt{{\mathrm{G}}_{\mathrm{x}}^2+{\mathrm{G}}_{\mathrm{y}}^2}}{\theta ={\tan }^{-1}\frac{{\mathrm{G}}_{\mathrm{y}}}{{\mathrm{G}}_{\mathrm{x}}}}}\end{array}$$

(37)

The LaneRTD (Figure 12) method encounters important limitations: it mainly detects straight lane segments, which results in poor representation of curved lanes unless additional parameters are introduced to handle curvature. Experimental evaluation confirms this behavior: while detection accuracy on straight lanes remained above 90%, it dropped below 65% on curved lanes. Furthermore, adjacent lanes in multi-lane scenarios were frequently missed, with omission rates approaching 50% in our tests. Performance degradation was also observed under occlusions caused by vehicles and shadow patterns, where the false negative rate increased by nearly 40%. These findings highlight the need for more robust parameterization and feature modeling to improve stability under diverse driving conditions.

The segmentation method [119], utilizing the Hough Transform, was tested on sample pavement images (Figure 13), leveraging its intensive use for line detection by assessing connections between random point pairs and selecting those exceeding a user-defined voting threshold.

For curved roads, the assumption remains valid, as only a 20 m segment in the near field was focused on. The transformation matrix was calculated to ensure that the resulting images approximately encompass the widest part of the pavement along the route and span a length of 20 m, enabling significant reductions in roadside facilities and plants (Figure 14).

The performance of lane detection methods depends strongly on the tuning of their parameters. Proper adjustment of discretization, smoothing, and thresholding ensures a balance between robustness, accuracy, and computational efficiency.

Detection of Traffic Signs

Traffic signs are categorized into three distinct groups based on geometric parameters that delineate their shapes: rectangles, circles, and triangles. The Hough transform, a dependable shape detection algorithm, is commonly employed due to its robustness against variations in illumination and partial obstructions. T. Bente [120] presented a driver assistance system in his paper, which is adept at identifying traffic signs through the use of an on-board camera. This system is also designed to alert drivers to potential lane deviations in videos recorded during daylight with moderately clear road markings. For the detection of traffic signs, the system utilizes an edge detection technique predicated on the Hough transform, as illustrated in Figure 15.

Compared to other authors, M. García-Garrido [121] proposed a method for detecting triangular-shaped traffic signs. The idea is to detect three intersecting straight lines forming an angle of 60 degrees. Note that, since the number of intersecting straight lines could be very large if the Hough transform were applied to the entire image, more lines than the actual triangles existing in Figure 16 would be detected in the real road, presented in Figure 16.

A limitation of this method is that we know neither the beginning nor the end of a straight line. This makes it necessary to apply the Hough transform to each contour, one after the other, which can increase the computation time in Figure 17.

The authors applied the same strategy for the detection of circular signs (including the stop sign). The estimation of the circular shape parameter is computed using the direction of the gradient of the studied contour according to [122]. Although the stop sign is octagonal, the difference between octagonal and circular signs is very small, and the former are also accepted. A circle in the (x, y) plane with center (χ, ψ) and radius ρ can be expressed according to the following Formula (38):

$$\begin{array}{c}(\mathrm{x}-\mathsf{\chi }{)}^2+(\mathrm{y}-\mathsf{\psi }{)}^2-{\mathsf{\rho }}^2=0\end{array}$$

(38)

Á. González [123] conducted an extensive examination of the geometries present in traffic signs to enhance detection accuracy. The algorithm’s goal is to precisely locate signs within images. This is achieved through an analysis of shapes derived from edge images. The Hough transform is applied to detect arrow-shaped, triangular, and rectangular panels, as illustrated in Figure 18. Edge detection is performed using Canny’s method, preserving closed contours as an essential aspect of leveraging shape information in traffic sign detection. Encoded contours obtained through the Canny method include area and perimeter information, facilitating the identification of closed contours. Subsequently, only accepted contours meeting these criteria undergo Hough transform applications, leading to reduced CPU computation time.

The parameters that define these shapes are used to reduce the search space using an accumulation matrix. Therefore, any estimate of the parameters of a straight line is calculated using the following Formula (39).

$$\begin{array}{c}{\displaystyle \genfrac{}{}{0pt}{}{\mathsf{\rho }=\frac{{\mathrm{x}}_1.{\mathrm{y}}_2 - {\mathrm{x}}_2{\mathrm{y}}_1}{\sqrt{{(\mathrm{x}}_1 - {\mathrm{x}}_2{)}^2+({\mathrm{y}}_1 - {\mathrm{y}}_2{)}^2}}}{\theta =\arctan \left(\frac{{\mathrm{x}}_1 - {\mathrm{x}}_2}{{\mathrm{y}}_1 - {\mathrm{y}}_2}\right)}}\end{array}$$

(39)

where $(x_1, y_1)$ and $\left(x_2, y_2\right)$ have points belonging to the studied contour. These estimated parameters determine the search area inside matrix a, i.e., the shaded area in Figure 19.

The search domain within the accumulator is confined; parameters pertaining to circular shapes are exclusively probed within designated shadow regions, as delineated in Figure 13. The pivotal advancement of this phase is the implementation of the Hough transform within constrained sectors of the accumulation matrix, maintaining precision. Employing this tactic enables the robust and real-time detection of diamonds, octagonal, and circular configurations, as evidenced in Figure 20.

The authors performed an assessment of detection and classification performances, outlined in

Table 8. Detection ratio for HT method.

Panel	Panel Detected (%)	Panel Validated (%)
STOP	99.92	98.92
Circles	99.92	98.92
Rectangles	99.46	96.79
Triangles	99.94	99.94
Total	99.81	99.64

, which presents the Hough detection ratios categorized by panel shapes. Panels were differentiated between detected and validated, with the latter requiring at least ten accurate detections. The overall detection rate achieved 99.81%, with a validation ratio of 98.10% for the signs. Particularly high ratios were observed for triangular, circular (STOP), and rectangular signs.

4.3.2. HOG Transforms

N. Romdhane [124] proposed a novel vision-based technique for detecting traffic signs. Initially, the method employs monocular segmentation, focusing on the color characteristics of candidate traffic signs. Subsequently, the Histogram of Oriented Gradients (HOG) operator is utilized to extract pertinent features. The results of this process are illustrated in Figure 21.

In the proposed method, every potential traffic sign region is standardized to 32 × 32 pixels. This region is then subdivided into non-overlapping local regions of size 12 × 12. From each of these local regions, Histogram of Oriented Gradient (HOG) features are extracted. This process involves computing histograms of edge gradients with nine orientations from each of the 4 × 4 local cells. Each pixel’s gradient is discretized within one of the nine orientation bins. Ultimately, the primary aim of the HOG is to encode the detected traffic signs and generate the feature vector. This vector is subsequently utilized as input for a Support Vector Machine (SVM) classifier to determine the class of traffic signs.

N. Romdhane [124] performed tests with a sample of the images resized to 288 × 384 pixels and comparisons to the literature methods to prove the performance of the proposed detection method. Detection requires about 0.957 s per image, which demonstrates its applicability in real-time systems, demonstrated in

Table 9. Treatment times.

Method	Treatment Times (s)
N. Romdhane [124]	0.957
J. Greenhalgh [126]	0.972
L. Chen [127]	0.984

. The HOG implementations were made on a GPU processor [125].

The reported processing time of 0.957 s per image for traffic sign detection raises questions about its applicability in autonomous driving, where rapid and reliable decision-making is crucial. Although the authors optimized the method of resizing images to 288 × 384 pixels and implementing the HOG on a GPU, this processing speed may not meet the stringent real-time requirements of dynamic driving environments. While suitable for moderate computational scenarios, further improvements are necessary for high-speed applications.

To address this, incorporating lightweight models like MobileNet or YOLO-Tiny could help achieve faster processing times. Additionally, leveraging hardware accelerators such as FPGAs or TPUs could further enhance computational performance. Streamlining the current GPU implementation by reducing the complexity of feature extraction or introducing batch processing could also contribute to improved efficiency. Future research should evaluate this approach under more realistic and demanding conditions, including high-resolution images and diverse road scenarios. By refining these aspects, the method could better align with the performance expectations of autonomous driving systems, ensuring timely and accurate traffic sign recognition for safe navigation.

4.4. Centroids and Contours

C. Tsai [128] dedicated their efforts to crafting a vision-based Advanced Driver Assistance System (ADAS) centered around the Speed Limit Sign (SLS) algorithm. This algorithm is engineered to automatically detect and identify speed limit signs on the road in real time, leveraging captured images. To enhance the SLS’s recognition capability across various orientations and sizes of signs within the images, the study introduced a novel algorithm. It delineates the recognized road sign by leveraging the distances between centroids and CtC (Center-to-Contour) contours of the extracted sign content. Known as the CtC description method, it utilizes CtC distances as shape features to describe a binarized image of the sign content. Figure 14 depicts the block function diagram of the proposed sign content method, named DeepL, which comprises six processing steps. Denoting the input binary image as I_b (x,y), the initial step involves calculating moments (Equation (40)) of the image of order {0, 1, 2}:

$$\begin{array}{c} {\mathrm{M}}_{\mathrm{p}\mathrm{q}}={\displaystyle \sum _{\mathrm{x}}}{\displaystyle \sum _{\mathrm{y}}}{\mathrm{x}}^{\mathrm{p}}{\mathrm{y}}^{\mathrm{q}}{\mathrm{I}}_{\mathrm{b}}(\mathrm{x},\mathrm{y})\end{array}$$

(40)

where p, q = 0, 1, 2. Then, the centroid of the binary image can be calculated using Equation (41):

$$\begin{array}{c} \overline{\mathrm{x}}={\displaystyle \frac{{\mathrm{M}}_{10}}{{\mathrm{M}}_{00}}} \mathrm{et} \overline{\mathrm{y}}={\displaystyle \frac{{\mathrm{M}}_{01}}{{\mathrm{M}}_{00}}}\end{array}$$

(41)

where $\overline{x}$ and $\overline{y}$ are the components of the centroid. On the basis of (42) and (43), three central moments of the second order, ${\mu }_{11}$, ${ \mu }_{20}$ and ${ \mu }_{02},$ can be determined by Equation (15):

$$\begin{array}{c}{\mathsf{\mu }}_{11}={\mathrm{M}}_{11}-\overline{\mathrm{x}} {\mathrm{M}}_{01} ; {\mathsf{\mu }}_{20}={\mathrm{M}}_{20}-\overline{\mathrm{x}} {\mathrm{M}}_{10} ; {\mathsf{\mu }}_{02}={\mathrm{M}}_{02}-\overline{\mathrm{x}} {\mathrm{M}}_{02}\end{array}$$

(42)

Then, the orientation of the binarized sign content is computed using the second-order central moments, such that (Equation (39))

$$\begin{array}{c}{\theta }_{\mathrm{s}\mathrm{i}\mathrm{g}\mathrm{n}}={\displaystyle \frac{1}{2}}\arctan \left({\displaystyle \frac{{2\mathsf{\mu }}_{11}}{{\mathsf{\mu }}_{20}-{\mathsf{\mu }}_{02}}}\right) \mathrm{for} {\mathsf{\mu }}_{20}-{\mathsf{\mu }}_{02}\ne 0\end{array}$$

(43)

where ${\theta }_{sign}$ is the orientation angle of the panel.

Figure 22 depicts the concluding pair of principles associated with the CtC descriptor technique. Let us define ($C_n$) as a contour trajectory encompassing (n) contour points, extending from coordinates ($(x_1,y_1)$) to $\left(x_n,y_n\right)$, where consecutive points $\left(x_i,y_i\right)$ and $\left(x_{i+1},y_{i+1}\right)$ are neighbors for 1 ≤ i ≤ n − 1. If we consider that (N) distinct contour trajectories have been extracted from the input binary image, for instance, N = 4 in the context of Figure 23, then for the j^ème contour trajectory $C_{n,j}^j$, with (j) ranging from 1 to (N), the computation involves determining the distance (Equation (44)) from the centroid of the image to each specific point along $C_{n,j}^j$:

$${\mathrm{d}}_{\mathrm{i}}^{\mathrm{j}}=\sqrt{({\mathrm{x}}_{\mathrm{i}}^{\mathrm{j}}-\overline{\mathrm{x}}{)}^2+({\mathrm{y}}_{\mathrm{i}}^{\mathrm{j}}-\overline{\mathrm{y}}{)}^2}$$

(44)

In this context, the coordinates $(x_i^j,y_i^j)$ for 1 ≤ i ≤ nj specify the position of a contour point on $C_{n,j}^j$. The term nj represents the count of contour points on $C_{n,j}^j$, and $d_i^j$ denotes CtC distance for each contour point on $C_{n,j}^j$.

The study introduces the CtC descriptor, which demonstrates robustness against translational, rotational, and scale variations observed in Speed Limit Signs (SLS) within images, as illustrated in Figure 23. This robustness substantially improves the accuracy of a support vector classifier that has been trained on a comprehensive traffic sign database. Upon deployment, the SLS recognition method functions efficiently on two separate embedded platforms, both powered by a Quad-Core ARM processor and operating under Android 4.4. The experimental results corroborate the efficacy of the proposed method, evidencing high recognition rates coupled with the capability for real-time processing. Moreover, the method attains real-time performance, achieving up to 30 frames per second while processing video streams at a resolution of 1280 × 720 on a commercially available ARM-based smartphone, the Asus PF500KL (Figure 23).

4.5. Comparative Analysis of Traffic Signs and Road Lane Detection Methods

Each author proposes a unique method for detecting road signs, underscoring the diversity within the field. Our paper’s primary focus lies in developing an efficient detector capable of real-time operation. To ensure the reliability and practicality of approaches, we will conduct an in-depth comparison of the geometric methods previously discussed. This comparison will evaluate their suitability for real-time applications concerning both traffic signs and roadways. The findings of this analysis will be meticulously documented in

Table 10. Detection method based on shape.

	Authors	Method	Dataset	Rate (%)	Times (ms)	False Alarms
Traffic Signs	F. Zaklouta [129]	HOG/SVM Linear	14,763 training images	90.90	55.54	-
	Ruta [20]	CDT Color Distance Transformer	13,287 images with radius between 15 and 25 pixels	90.30	-	9%
	M. García-garrido [130]	Hough Transform	Spanish code of conduct.	99	30	2%
	M. García-garrido [121]	Hough Transform	Triangular panel (Belgium Road Code)	94.2–97.3	20	-
	A. Youssef [100]	HOG	GTSDB	89.71–98.67	197–693	-
	J. Borrego-carazo [131]	CtC	42,413 images (German, Belgium)	99	30	-
Road Lane	Jungang Guan [132]	Hough Transform	Video (1920 × 1080 pixels) à 14.3 ms/image on average	99	5.4	-
	W. Farage [118]	Hough Transform + Canny	Video (960 × 540 pixels)	99	10	-
	A. Gupta [133]	Grassmann’s discriminant analysis	Video (320 × 240 pixels)	95	28–36.47	0.83
	M. Park [134]	HOG	Video (640 × 480 pixels)	88.19	51	-
	D. Y. Huang [135]	HOG-SVM	Video (320 × 240) pixels	94.08	N/A	-

. From our extensive study, we aim to discern the most effective strategy for constructing a robust detector capable of precisely identifying regions pertinent to road signs and road lines, including both right and left lanes, crucial for accurate lane detection. We hypothesize that a synergistic integration of color-based and geometric methods will prove to be the most efficacious approach in achieving conditions for realization vision system applicable for automotive.

4.6. Artificial Intelligence Methods

Learning methods, a category within machine learning techniques, have instigated a revolution across multiple industrial sectors, particularly within automotive embedded systems. Unlike traditional geometric and color-based approaches for object detection, learning methods offer substantial advancements. They mitigate challenges such as variations in brightness, temporal changes, scale adjustments, and object rotations. At the core of the effectiveness of learning methods lies the availability of extensive datasets and robust hardware infrastructure.

Detecting and recognizing traffic signs in the road environment poses a significant challenge. Gudigar [136] tackled this challenge by leveraging higher-order spectra (HOS) in conjunction utilizing characteristics based on texture to construct a structured traffic sign recognition (TSR) model, as depicted in Figure 24. These features offer a clear representation of both the form and content of road signs. Subsequently, a learning method employing K-NN architecture integrated with Linear Discriminant Analysis (LDA) graph subspaces within the linear discriminant analysis framework was employed to enhance discrimination power among various traffic symbols.

LDA applies covariance maximization between classes and minimizing covariance within a class. In practice, we have S training signs ${\left\{X_j\right\}}_{j-1}^S\subset {\mathbb{R}}^D, X=[x_1,x_2,x_3,x_4\dots .,x_S]$, with c classes, and the kth classes have S_k samples. The weight matrix for LDA can be described by incorporating the label information according to the following function (45):

$$W_{j,k}=\left\{\begin{array}{c}{\displaystyle \frac{1}{S_k}}, if x_j and x_k share the label k\\ 0, otherwise\end{array}\right.$$

(45)

In

Table 11. Accuracy of the TSR approach.

Method	Technique	Dataset	Precision (%)
HOS and K-NN	--	BTSC	98.89
		GTSRB	97.84
	LDA	BTSC	97.90
		GTSRB	97.47

, the method presented in this study attained a peak recognition accuracy of 98.89%. This evaluation was conducted using two commonly accessible datasets: BTSC and GTSRB. However, there exists a notable absence of experimental results that sufficiently validate the effectiveness and reliability of the proposed method in achieving the specified recognition accuracy level.

On the other hand, the author G. Villalón-Sepúlveda [137] made an analysis of the detection rate as a function of the distance between the vehicle and the panel by proposing a statistical model detection approach (Figure 25).

To validate the effectiveness of the current methodologies, tests were conducted using a training dataset consisting of 2567 images, among which 122 contained stop signs and 80 contained prohibitory signs. Additionally, a validation dataset was employed, comprising 273 stop signs and 447 no-passing signs. These datasets encompass a series of signs encountered at various intersections in Santiago, Chile, capturing real-world driving conditions marked by variable lighting and partial obstruction. The dataset has been made accessible to the Robotics and Automation Laboratory (RAL) at the School of Engineering, Pontificia Universidad Católica de Chile, to support further research and development in Advanced Driver Assistance Systems (ADASs).

The authors did not manage to detect signs for distances greater than 62 m or less than 20 m. Figure 26 shows that the Viola–Jones approach is very sensitive to possible changes in the signs searched.

Ellahyani [138] showcased the efficacy of combining color and shape details for traffic sign recognition. Initially, the input image undergoes preprocessing using the mean shift clustering algorithm, which clusters relying on color information. Then, a Random Forest classifier segments the clustered image, as depicted in Figure 27. Notably, the mean shift clustering algorithm relies on two key parameters: spatial resolution (SR) and color resolution (CR).

SR = 30 and CR = 70 were chosen for their ability to ensure high accuracy, with over 297 true positives, while simultaneously maintaining minimal computation time, under 50 ms. The results of mean shift clustering as SR and CR vary are depicted in Figure 27.

Next, a log-polar transform is utilized for shape-based classification [139], coupled with the cross-correlation technique. This approach operates within polar coordinates, encompassing both the radial distance from the center and the angle. Log-polar coordinates are derived from the logarithm of the radial distance, allowing any point (x, y) to be expressed in polar coordinates (ρ, θ) as detailed in the equation below (46):

$$(\mathsf{\rho },\theta )=(\sqrt{(\mathrm{y}-{\mathrm{y}}_{\mathrm{c}}{)}^2+(\mathrm{x}-{\mathrm{x}}_{\mathrm{c}}{)}^2},\arctan \left({\displaystyle \frac{\mathrm{y}-{\mathrm{y}}_{\mathrm{c}}}{\mathrm{x}-{\mathrm{x}}_{\mathrm{c}}}}\right))$$

(46)

In this scenario, ρ represents the radial distance from the image’s center, while θ denotes the angle. The coordinates (x_c, y_c) signify the center of the window. Upon applying the conversion of polar coordinates into an image within the Cartesian domain, radial lines in the Cartesian domain are transposed into horizontal lines in the (ρ, θ) domain. As a result, rotations are transformed into cyclic translations along the θ-axis, whereas scaling is converted into translations along the ρ axis. Figure 17 serves as an illustration of a log-polar transformation. Notably, scaled and rotated patterns are transmuted into shifted patterns in log-polar space [140].

In this scenario, the variable ρ denotes the radial distance from the central point of the image, while θ represents the angular component. The coordinates (x_c, y_c) define the center of the window of interest. The transformation from polar to Cartesian coordinates within an image results in the conversion of radial lines in the Cartesian plane into horizontal lines within the (ρ, θ) domain. Consequently, rotational movements are expressed as cyclic translations along the θ-axis, and scaling variations manifest as translations along the ρ-axis. Figure 28 exemplifies a log-polar transformation, where it is observed that patterns subjected to scaling and rotation are effectively transformed into shifted patterns within the log-polar space, as referenced in [67].

Figure 29 exhibits the patches utilized in this study in comparison with the extracted regions of interest (ROIs) and their corresponding log-polar transformations. Cross-correlation is applied to assess the similarity between the log-polar transformations of the ROIs obtained through the segmentation step.

The approach of detection proposed in this study underwent testing on both the GTSDB and STS datasets (Figure 30); the method achieves scores of 93.50% and 94.22% in terms of F-1 and AUC (Area Under the Curve) measurements, respectively, in

Table 12. The best values of the log-polar transformation method.

Method	Dataset	Precision (%)	Rappel (%)	AUC (%)
Transformation Log-polar	STS	94.15	93.87	95.17
	GTSDB	94.03	92.98	94.22

. Precision and recall parameters [141] are calculated using Equations (47) and (48). F_N, F_P, T_P, and T_N are the false negative, false positive, true positive, and true negative samples, respectively. The Area Under the Receiver Operating Characteristic Curve (AUC) serves as a crucial metric for evaluating a computer vision model’s performance across different classification thresholds. This index holds significance as it provides a quantifiable measure of the model’s predictive capability. Essentially, the AUC reflects how well the model distinguishes between classes, making it an essential tool in assessing the overall effectiveness and discriminatory power of the computer vision system.

$$\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}={\displaystyle \frac{{\mathrm{T}}_{\mathrm{P}}}{{\mathrm{T}}_{\mathrm{P}}+{\mathrm{F}}_{\mathrm{P}}}}$$

(47)

$$\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}={\displaystyle \frac{{\mathrm{T}}_{\mathrm{P}}}{{\mathrm{T}}_{\mathrm{P}}+{\mathrm{F}}_{\mathrm{N}}}}$$

(48)

Accurate and robust lane recognition holds significant importance for the progress of autonomous vehicles in the foreseeable future. L. Kalms [142] introduced a robust algorithm utilizing the Viola–Jones method for lane recognition. The Viola–Jones method is deployed to detect traffic cones positioned off the “emergency case” road, as depicted in Figure 31. This paper [142] places particular emphasis on the setting of the steering wheel angle.

The arrangement of traffic cones is meticulously analyzed to construct an accurate road model, which facilitates the autonomous and secure maneuvering of vehicles in exigent circumstances. This methodology is implemented on a Raspberry Pi apparatus and its efficacy is evaluated through a driving simulation. In the context of high-definition imagery, with a resolution of 1920 × 1080 pixels, the temporal requirement for object detection is achieved in under 218 milliseconds, while maintaining a commendable detection accuracy. Furthermore, the computational demands for the formulation and actualization of autonomous vehicular control are remarkably minimal, necessitating merely 0.55 milliseconds

Therefore, according to the methods and strategies discussed above, we make a comparison of accuracy and time for detection and recognition, either for road signs or for road lanes, in

Table 13. Detection and recognition of road lanes.

Road lane	Authors	Method	Precision (%)	Times (ms)
	Ö. Kumtepe [143]	Viola–Jones	90	--
	R. Anderson [144]	Viola–Jones and YOLO	--	14.8
	W. Song [145]	Hough + CNN	99.6	--

and

Table 14. Detection and recognition of traffic signs.

	Authors	Method	Dataset	Traffic Signs			Precision (%)	Times (ms)
				Vitesse Limits (%)	Interdiction (%)	Danger (%)
Traffic Signs	J. Zhang [146]	HOS-LDA	GTSRB	99.93	99.44	99.13	97.84	0.64
	Ruta [147]	HOG + KELM	GTSRB BTSC MASTIF	99.54	100	98.96	98.56	3.9
	M. Liang [148]	HOG + Color	GTSDB	86.91	92	86.34	89.49	--
	L. Abdi [149]	Viola-Jones	GTSRB	44.87	90.81	46.26	64.66	--
	A. Jose [150]	Viola-jones + CNN	GTSRB	94.10	--	21.43%	90%	--
	A. Gudigar [151]	Log-polar	GTSDB GTSRB	98.46	98.55	98.62	98.31	0.40
	A. Ellahyani [152]	Log-polar	GTSRB	--	--	--	97.96	51.35

.

4.7. Recognition Methods

In this section, we will discuss one of the techniques that plays a key role in the development of intelligent transportation systems. Techniques that use manual methods to extract features from traffic signs, as well as voices from the road, are combined with an intelligent phase to recognize objects, such as deep learning to detect and recognize objects at the same time.

4.7.1. Learning Methods Based on Manually Extracted Features

S. Azimi [153] introduced an advancement in the form of wavelet-transform-augmented fully convolutional symmetric neural networks (FCNNs), specifically engineered for the autonomous segmentation of lane markings in aerial imagery via the discrete wavelet transform (DWT). The computation of the DWT for a signal x involves the application of a sequence of filters and subsequent subsampling across multiple scales, as depicted in Figure 32. To tackle the pronounced imbalance between the quantity of lane marking pixels and background pixels, a bespoke loss function was devised, complemented by an innovative data augmentation technique. This methodology has been demonstrated to achieve superior accuracy in the pixel-level identification of lane markings, outperforming contemporary methods, while independently operating without reliance on external data sources.

In the manuscript in question, the researchers delineate a robust and expedient segmentation algorithm tailored for diminutive entities, such as traffic lane demarcations in aerial imagery, achieving an impressive accuracy of 99.81% alongside substantial resilience. This is accomplished through the deployment of an Aerial LaneNet predicated on a fully convolutional neural network (FCNN) framework. The novel integration of Discrete wavelet transform (DWT) coefficients to augment FCNNs for pixel-wise semantic segmentation affords a comprehensive multispectral and multiresolution examination. This enhancement precipitates a marked augmentation in performance relative to the linear FCNN paradigms, as depicted in Figure 33.

In their research, S. Aziz [154] proposed an innovative and efficient technique for traffic sign recognition by integrating complementary and discriminative feature sets. These features include the Histogram of Oriented Gradients (HOG), the Gabor feature, and the compound local binary pattern (CLBP), as depicted in Figure 34.

In the delineated methodology, referred to as compound local binary patterns (CLBPs), feature extraction is predicated on a refined variant of local binary patterns (LBPs). A 2-bit coding scheme is utilized to encapsulate the local textural attributes of an image. The initial bit is indicative of the signum function applied to the disparity between the center pixel’s value and that of its adjacent pixel. Concurrently, the second bit quantifies the magnitude of this difference, benchmarked against a threshold termed $M_{avg}$. This threshold is ascertained by computing the mean magnitude of the disparities between the central pixel and its neighbors within the immediate vicinity. Denoting a neighboring pixel as $i_p$ and the central pixel as $i_c$, the threshold $M_{avg}$ is derived as the average magnitude of the differences between $i_p$ and $i_c$, in the local neighborhood (Figure 34). The binary code S(x) is then formulated according to the subsequent Equation (49):

$$\mathrm{S}\left({\mathrm{i}}_{\mathrm{p}},{\mathrm{i}}_{\mathrm{c}}\right)=\left\{\begin{array}{c}00 {\mathrm{i}}_{\mathrm{p}}-{\mathrm{i}}_{\mathrm{c}}<0, |{\mathrm{i}}_{\mathrm{p}}-{\mathrm{i}}_{\mathrm{c}}|\le {\mathrm{M}}_{\mathrm{a}\mathrm{v}\mathrm{g}}\\ 01 {\mathrm{i}}_{\mathrm{p}}-{\mathrm{i}}_{\mathrm{c}}<0, \left|{\mathrm{i}}_{\mathrm{p}}-{\mathrm{i}}_{\mathrm{c}}\right|>{\mathrm{M}}_{\mathrm{a}\mathrm{v}\mathrm{g}}\\ 10 {\mathrm{i}}_{\mathrm{p}}-{\mathrm{i}}_{\mathrm{c}}\ge 0, \left|{\mathrm{i}}_{\mathrm{p}}-{\mathrm{i}}_{\mathrm{c}}\right|>{\mathrm{M}}_{\mathrm{a}\mathrm{v}\mathrm{g}}\\ 11 \mathrm{a}\mathrm{u}\mathrm{t}\mathrm{r}\mathrm{e}\end{array}\right.$$

(49)

The Extreme Learning Machine (ELM) algorithm is employed alongside the compound local binary pattern (CLBP) for classification (Figure 35). This innovative approach integrates complementary and informational features to enhance traffic sign recognition. The effectiveness of this method is evaluated using the German Traffic Sign Recognition Benchmark (GTSRB) and the Belgian Traffic Sign Classification (BTSC) datasets.

The experimental results presented in

Table 15. Comparison of roads line recognition methods.

Dataset	Method	Precision (%)	Time (s)
GTSDB	HOG	95.70	0.08
	CLBP	96.88	1.21
	Gabor	94.10	2.32
	HOG + CLBP	97.03	1.49
	HOG + Gabor	96.90	2.57
	CLBP + Gabor	96.40	3.54
	HOG + CLBP + Gabor	99.10	3.68
BTSC	HOG	94.98	0.06
	CLBP	95.50	1.18
	Gabor	93.18	2.09
	HOG + CLBP	96.58	1.27
	HOG + Gabor	96.74	2.42
	CLBP + Gabor	97.04	3.40
	HOG + CLBP + Gabor	98.30	3.50

demonstrate that each feature individually achieves relatively high accuracy. However, the combination of three features (HOG + CLBP + Gabor) exhibits excellent complementarity, resulting in a rapid recognition rate of nearly 98%. This swift recognition rate renders it highly suitable for real-time applications. Z. Malik [155] presented an efficient technique to extract road signs from images. Detection involves segmenting based on color and subsequently utilizing the Hough transform to identify geometric shapes like circles, triangles, or rectangles. Recognition is achieved through three state-of-the-art feature matching techniques: Scale-Invariant Feature Transform (SIFT), Speeded-Up Robust Features (SURFs), and Binary Robust Invariant Scalable Key (BRISK) points.

Illustrated in Figure 36 is the comparison of an exemplar road sign utilizing the three descriptors, while

Table 16. Comparison of road sign recognition methods.

Algorithm	Temps	Scenario I	Scenario II
SIFT	2.1 t	100%	93.75%
BRISK	1.4 t	93.75%	87.5%
SURF	t	93.75%	81.25%

presents a summary of the recognition rates. It is essential to highlight that recognition is performed under two distinct scenarios. The system was developed using Matlab, with an average processing time of 4.3 s on an Intel i3 machine equipped with 4 GB of RAM.

In the first scenario, the recognition performance is evaluated by comparing manually segmented road signs with those in the database. This approach assesses how accurately the system recognizes signs when provided with perfectly segmented images. In the second scenario, the detection system’s output, which may not perfectly segment road signs, is directly matched with signs in the database to identify real-world matches. This scenario evaluates the system’s ability to recognize signs in practical, imperfect conditions encountered in real-world settings.

The analyzed system, evaluated with a custom dataset, demonstrated favorable results in both detection and recognition tasks.

Upon comparative analysis of the three descriptors, it was found that the SIFT achieves the highest recognition rates, whereas the BRISK stands out as the most efficient in terms of computation time (refer to Figure 37). It is important to note that the system is specifically designed for processing daytime images. To enhance its functionality, future endeavors could focus on extending the system’s capability to detect and recognize traffic signs from images captured at night. While color-based segmentation provided acceptable detections, exploring more sophisticated techniques could offer valuable insights for further improving performance.

P. Dhar [156] designed and developed a traffic sign recognition (TSR) system specifically tailored for Bangladesh signs. The system leverages color cues and employs a CNN as an extractor. The algorithm, depicted in Figure 38, follows these steps:

• Acquisition: The input image is acquired.
• Pre-processing: Initial preprocessing tasks are performed.
• Segmentation: The image is segmented using color information from the HSV color model.
• Morphological Closure: Refinement of the segmented image occurs.
• Region Filtering: Filtering based on region properties and shape signature is applied.
• Region Cropping: The desired region (likely containing the traffic sign) is cropped.
• Classification: The extracted sign region undergoes classification using automatic feature extraction via the deep CNN.

This system aims to enhance traffic safety by accurately recognizing and classifying road signs.

The empirical findings underscore the commendable efficacy of the algorithm under scrutiny, as evidenced by its substantial recognition precision. Subsequent to feature extraction, the convolutional neural network (CNN) is subjected to a training regimen, which then enables it to categorize images based on the aforementioned features. The experimental setup was calibrated to a ceiling of 20 epochs, with an inaugural learning rate established at 0.0001. The computation of the mini-batch loss is executed via the cross-entropy function across K distinct classes, and the mini-batch accuracy metric reflects the ratio of accurately classified images within the current mini-batch by the trained network. The metric of accuracy is quantified as a percentage, derived from the ratio of the true positive rate to the aggregate count of images.

The proposed system achieves a significant classification accuracy of 97%, representing a noteworthy and competitive accomplishment. When evaluating the algorithm against alternative learning methods, the Speeded-Up Robust Feature (SURF) descriptor is utilized for feature creation. SURFs, a derivative of the SIFT, serve as both key point feature detectors and descriptors. Key points are identified based on the Hessian matrix with Gaussian differential (DoG), while descriptors are generated from Haar wavelet responses centered on these key points and their surrounding areas.

Table 17. Accuracy of classification using SURF method.

Classifier	Method	Precision
SVM (Cubic)	SURF	82.0%
SVM(Quadratic)		81.0%
ANN		93.3%
KNN		76.0%
Decision Trees		71.0%
Ensembles (Adaboost)		68.0%
CNN (Proposed)	CNN Extractor	97.0%

illustrates the classification accuracy achieved using SURFs across various classifiers. Upon thorough examination of the results, it is apparent that CNN outperforms other classifiers in terms of performance.

4.7.2. Deep Learning Methods

Indeed, traffic sign detection systems are integral components of various real-world applications, including autonomous driving, safety systems, and driver assistance technologies. These systems play a crucial role in ensuring road safety, enhancing driver awareness, and facilitating efficient navigation on roads. By accurately detecting and interpreting traffic signs, these systems contribute significantly to improving overall road safety and enhancing the driving experience for individuals. Á. Arcos-García [157] evaluated various object detection systems, including SSD, the Faster R-CNN, YOLO V2, and the R-FCN, combined with various feature extractors, such as ResNet V1 50, ResNet V1 101, Inception V2, MobileNet V1, and DarkNet-19. This evaluation was conducted over time, as depicted in Figure 39.

The objective was to investigate the attributes of these object detection models, specifically adapted for the task of traffic sign detection. Transfer learning is primarily employed for this purpose. Specifically, to adopt object detection models pre-trained on the Microsoft COCO dataset for use with the GTSDB dataset, several adjustments are typically made. The mean Average Precision (mAP) calculation involves averaging the individual Average Precisions obtained for each class, as depicted in Figure 40.

The precision for a given recall point ($r_1$) is normalized by dividing it by the maximum precision achieved at any recall point $r_2$ greater than or equal to $r_1$, as per Equation (2X). Here, p($r_1$) represents the measured precision at recall $r_1$. The cumulative precision, calculated at each point i where there is a change in recall, is multiplied by ∆$r_1\left(i\right)$, the corresponding change in recall, resulting in the overall sum [157].

Frames per second (FPS) is a measure that signifies the quantity of frames displayed on the screen. The higher the FPS, the more seamless the motion appears. In the context of this experiment, FPS is employed to classifier ability to achieve high mAP (Equations (50) and (51)).

$$\mathrm{p}\left({\mathrm{r}}_1\right)={\mathrm{m}\mathrm{a}\mathrm{x}}_{{\mathrm{r}}_2{:{\mathrm{r}}_2>\mathrm{r}}_1}\mathrm{p}\left({\mathrm{r}}_2\right)$$

(50)

$$\mathrm{A}\mathrm{P}=\sum _{\mathrm{i}=1}^{\mathrm{N}}\mathrm{p}\left(\mathrm{k}\right).∆{\mathrm{r}}_1(\mathrm{i})$$

(51)

The comparative analysis of these computational models is predicated upon a multitude of critical metrics, such as average memory allocation, floating-point operations count, precision (measured as mean Average Precision, mAP), shown in

Table 18. Properties of the models classified by average accuracy (mAP).

Classifications	mAP (%)	FPS	Belgique (MB)
Faster R-CNN Inception ResNet V2	95.77	2.26	18,250.45
R-FCN ResNet 101	95.15	11.70	3509.75
Faster R-CNN ResNet 101	95.08	8.11	6134.71
Faster R-CNN ResNet 50	91.52	9.61	5256.45
Faster R-CNN Inception V2	90.62	17.08	2175.21
SSD Inception V2	66.10	42.12	284.51
SSD MobileNet	61.64	66.03	94.70

, parameter quantity, computational time, and the influence of the dimensions of traffic sign imagery. The findings reveal that R-CNN Inception, augmented with Faster ResNet V2, attains the apex in mAP. Concurrently, R-FCN ResNet 101 emerges as the paradigm of equilibrium between precision and computational expediency. Distinctively, YOLO V2 and SSD MobileNet are prominent; the antecedent secures commendable precision and is distinguished as the penultimate in detection rapidity, whereas the latter is celebrated as the preeminent model in terms of minimal memory requisites. Consequently, SSD MobileNet is deemed the quintessential candidate for integration within mobile and embedded apparatuses.

Álvaro Arcos-García [158] presented a novel deep learning approach tailored for traffic sign recognition systems. Through a series of classification experiments conducted on publicly available datasets of Belgian traffic signs, the proposed methodology leverages a DL NN architecture incorporating convolutional layers and spatial transformation networks, as depicted in Figure 41. Initially, the algorithm’s performance is evaluated using different optimizers, including SGD, RMSprop, and Adam, as illustrated in the following Figure 42. The convolutional neural network developed achieved an impressive recognition rate of 99.71% accuracy in tests.

The experiments are designed to evaluate the influence of different variables, aiming to develop a convolutional neural network that enhances the accuracy of traffic sign recognition, as detailed in

Table 19. Reported precision and recall for traffic sign recognition methods.

	Authors	Dataset	Method	Precision (%)	Recall (%)
Learning methods based on manually extracted features	Abedin et al. [49]	200 images	SURF + ANN	----	97
	Malik et al. [14]	172 images	SIFT SURF BRISK	---	93.75 81.25 87.50
Deep learning methods	A. Haghighat [159]	GTSDB	Architecture CNN	99.4	--
	X. BANGQUAN [160]	GTSRB et LISA US	Architecture CNN	96.80	--

.

Hierarchical neural networks have demonstrated efficacy in learning features from images and classifying object classes. However, many existing networks focus solely on integrating low-to-medium-level cues for classification, overlooking spatial structures. In tasks such as scene understanding, the arrangement of visual cues within an image’s spatial distribution significantly impacts accurate analysis, especially in tasks like detecting roadway edges, as outlined in

Table 20. Detection and recognition of the lines of the road.

	Authors	Dataset	Method	Precision (%)	Recall (%)
Learning methods based on manually extracted features	Lingni Ma [161]	Cityspaces	DWT + CNN	92.80	---
	L. Abdi [162]	GTSRB	Cascade Haar + CNN	98.81	98.22
Deep learning methods	Z. Qin [163]	TuSimple CULane	Architecture RNN	96.06	--
	Q. Zou [164]	TuSimple Lane	CNN + RNN	97.30	90.50

.

4.8. Impact of Lighting Conditions on Traffic Sign Detection: Methodological and Dataset Analysis

Variations in lighting, such as low light during nighttime, intense sunlight, and shifting shadows, present critical challenges for traffic sign detection and recognition systems. These environmental factors significantly affect detection accuracy, especially in dynamic real-world settings where lighting conditions constantly change. To overcome these limitations, it is essential to examine both the detection methods and the datasets used to train these models, ensuring their robustness and adaptability.

The following table,

Table 21. Diversity under lighting conditions.

Method	Key Strength	Weakness	Performance Accuracy (%)	Processing Time (ms)	Positive False Rate (%)	Robustness to Occlusion	Hardware Dependency
Adaptive Image Enhancement [165]	Improves image quality in low light and glare	Limited for extreme lighting changes	85	30	10	Moderate	Moderate
Color-Based Segmentation [108]	Effectively isolates signs under low-light conditions	Sensitivity to noise and shadows	80	25	12	Low	Low
Photometric Invariants [166]	Handles illumination variations robustly	Requires high computational resources	88	40	8	High	High
YOLO-Based Models [167]	High detection accuracy and speed	Performance depends on training data quality	92	15	6	High	High
Combined Transfer Learning and YOLO [168]	Improved detection in diverse lighting conditions	Needs more training data for optimization	91	20	5	High	High

, provides a concise overview of this issue, summarizing key performance indicators for different detection methods and datasets. By analyzing detection accuracy, false positive rates, and the diversity of dataset conditions, these tables shed light on how lighting variability impacts the effectiveness of traffic sign and road lane recognition systems.

The comparison of detection methods under various climatic and lighting conditions reveals notable performance differences. Advanced approaches, such as YOLO-based models, demonstrate a high detection accuracy of 92% with a low false positive rate of 6%, showcasing their reliability in challenging scenarios like shadows, glare, and low-light conditions. Similarly, the combined transfer learning and YOLO method achieves an accuracy of 91% and a false positive rate of 5%, reflecting its ability to generalize effectively across diverse datasets. In contrast, simpler techniques like adaptive image enhancement and color-based segmentation show lower accuracies of 85% and 80%, respectively, due to their sensitivity to extreme lighting changes. These findings highlight the advantage of deep learning-based techniques in maintaining precision and minimizing errors in real-world applications.

The analysis of datasets emphasizes the importance of diversity in enhancing detection performance (

Table 22. Comparison of datasets and their diversity in lighting conditions.

Dataset Name	Number of Images	Lighting Scenarios	Resolution Range	Annotation Type	Diversity in Sign Types	Geographical Coverage	Scientific Reference
GTSRB	51,839	Normal daylight, limited low-light data	30 × 30 to 256 × 256	Bounding boxes	High	Germany	https://benchmark.ini.rub.de/ (accessed on 11 June 2025)
Cityscapes	25,000	Daylight with some shadow variations	2048 × 1024	Semantic segmentation	Moderate	Global (urban-focused)	https://www.cityscapes-dataset.com/ (accessed on 11 June 2025)
TTSDCE	1800	Daylight and low light with some urban conditions	300 × 300 to 1280 × 720	Bounding boxes	Moderate	China and English regions	N/A (self-collected)
KITTI	14,999	Varied conditions including shadows and glare	Varied (~1242 × 375)	Bounding boxes and lanes	Low	Germany	http://www.cvlibs.net/datasets/kitti/ (accessed on 11 June 2025)
Mapillary Traffic Sign Dataset	30,000	Highly diverse lighting scenarios	Varied (~1920 × 1080)	Bounding boxes, semantic segmentation	High	Global	https://arxiv.org/abs/1909.04422 (accessed on 11 June 2025)

). For instance, the Mapillary Traffic Sign Dataset, with its wide range of lighting conditions, supports the development of robust detection models that perform well under diverse climatic scenarios. Conversely, datasets like GTSRB, primarily focused on daylight settings, offer limited adaptability to varying conditions. Cityscapes and KITTI datasets, designed for urban and highway environments, provide valuable annotations that enable algorithms to handle specific use cases effectively, such as lane detection and traffic sign recognition in challenging weather conditions. By utilizing datasets with broader coverage, like Mapillary, detection methods can achieve greater accuracy and resilience, ensuring reliability in dynamic and unpredictable real-world environments.

The ability of traffic sign detection models to generalize effectively is heavily influenced by the diversity of datasets used for training. Many widely used datasets, such as GTSRB, with over 51,000 images spanning 43 common sign categories, fall short when it comes to rare or region-specific traffic signs. Similarly, KITTI, containing 14,999 images focused on urban and highway environments, lacks sufficient coverage of less frequent signs. This imbalance in dataset representation limits the adaptability of models, reducing their effectiveness in recognizing uncommon traffic signs in diverse real-world conditions.

On the other hand, datasets like the Mapillary Traffic Sign Dataset offer broader coverage, featuring 100,000 images that include more than 200 unique traffic sign classes across varying lighting and geographic conditions. Such datasets enhance the robustness of detection systems by enabling them to learn from a richer variety of signs. Incorporating synthetic data generation and transfer learning techniques can further address the scarcity of rare sign examples, expanding dataset diversity. These approaches are vital to building models that achieve high accuracy and reliability across both common and rare traffic signs, ensuring they perform consistently in dynamic and complex environments.

4.9. Parameter Tuning and Its Impact on Model Performance

The effectiveness of traffic sign detection models is significantly influenced by their parameter tuning, which can directly affect convergence and accuracy. Fine-tuning parameters such as learning rate, batch size, and regularization factors is crucial for optimizing performance. However, improper tuning may lead to early convergence, limiting the ability of models to generalize in environments with complex variations. For instance, in YOLO-based models, an overly aggressive learning rate can result in suboptimal detection accuracy, especially under conditions involving shadow variations or occlusions [169]. Recent studies have proposed adaptive tuning strategies to address these challenges. For example, the Faster R-CNN model incorporates region proposal networks (RPNs), which dynamically adjust parameters based on feature extraction, achieving an accuracy of 92.0% even in challenging environments [170]. Similarly, SSD with an FPN leverages features pyramids for improved scale invariance, maintaining a detection accuracy of 93.8% while reducing false positives to 6.2% [171].

Hyperparameter tuning is crucial for optimizing machine learning models in traffic sign detection and other applications. By selecting the optimal combination of hyperparameters, these models can achieve superior performance while minimizing computational costs. Techniques such as Bayesian Optimization, Gradient Descent, and meta-learning have been employed to refine this process, leveraging mathematical frameworks for effective parameter selection. Bayesian Optimization has been widely utilized to streamline hyperparameter tuning by approximating the objective function f(x) through a surrogate model S(x), often a Gaussian process. The acquisition function A(x) is used to identify the next promising hyperparameter configuration (function (52)):

$${\mathrm{x}}_{\mathrm{n}\mathrm{e}\mathrm{x}\mathrm{t}}=\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{m}\mathrm{a}\mathrm{x}\mathrm{A}\left(\mathrm{x}|\mathrm{S}\left(\mathrm{x}\right),\mathrm{D}\right)$$

(52)

where D represents the dataset of previously evaluated configurations [172]

Gradient-based optimization, such as Stochastic Gradient Descent (SGD: Equation (53)), iteratively adjusts hyperparameters like weights and learning rates:

$${\theta }_{\mathrm{t}+1}={\theta }_{\mathrm{t}}-\mathsf{\eta }.{\nabla }_{\theta }\mathcal{L}(\theta )$$

(53)

where η is the learning rate, and ${\nabla }_{\theta }\mathcal{L}\left(\theta \right)$ is the gradient of the loss function [173]

Additionally, regularization plays a pivotal role in reducing overfitting during tuning. The regularized loss function is expressed (54) as

$$\mathcal{L}(\theta ){\theta }_{\mathrm{t}+1}=({\mathcal{L}}_0(\theta )-\mathsf{\lambda }{\Vert \theta \Vert }^2)$$

(54)

where λ controls the strength of regularization [174]. The mathematical frameworks and strategies demonstrated in these articles highlight the pivotal role of hyperparameter tuning in optimizing traffic sign detection models. By employing Bayesian Optimization, Gradient Descent, and meta-learning, these studies illustrate the balance between performance and computational efficiency, addressing challenges in complex real-world applications.

Table 23. The hyperparameter tuning in traffic sign detection models.

Optimization Method	Number of Iterations	Impact on Accuracy (%)	Impact on Computational Cost (%)
Pelican Optimization Algorithm (POA) and Cuckoo Search Algorithm (CSA) [175]	1000	4.5	−15%
Bayesian Optimization with Angular Margin Loss [176]	1200	5.2	−12%
Bayesian Optimization [177]	800	3.8	−10%
ShuffleNet with YOLOv5 tuning [178]	950	4.8	−18%
Reparameterized YOLOX-s [179]	1100	5	−16%
Crowdsourced Hyperparameter Tuning [180]	Variable	6	−20%
Automated Hyperparameter Search [181]	1500	4.2	−14%
Adaptive Hyperparameter Selection [182]	900	4.7	−13%
Meta-Learning Hyperparameter Tuning [183]	1300	5.5	−17%
Sensitivity Analysis with Hyperparameter Adjustment [184]	1100	4	−11%

provides a detailed overview of various studies focusing on hyperparameter tuning in traffic sign detection models, highlighting the optimization methods employed, the number of iterations involved, their impact on accuracy, computational cost, and direct links to the respective articles for in-depth reading.

The detailed analysis of hyperparameter tuning in traffic sign detection models offers valuable insights into optimization techniques, their impact on accuracy, and computational efficiency. Table 23 provides a structured comparison of various strategies, highlighting trends that inform the development of high-performing detection systems.

Bayesian Optimization stands out as a dominant technique, appearing in multiple studies with different configurations. For instance, the “Bayesian Angular Margin Loss” approach achieves an accuracy improvement of +5.2% while reducing computational cost by −12%, requiring approximately 1200 iterations. This balance of performance and optimization effort makes it well suited for systems with moderate computational resources. In contrast, optimization methods such as the Pelican Optimization Algorithm (POA) and Cuckoo Search Algorithm (CSA) show accuracy gains of +4.5% and cost reductions of −15% after 1000 iterations. These approaches offer a practical trade-off between efficiency and performance, particularly advantageous in resource-constrained environments. A standout technique is Crowdsourced Hyperparameter Tuning, which achieves the highest accuracy improvement of +6.0% and a significant cost reduction of −20%. By leveraging distributed optimization processes, this method demonstrates potential for large-scale applications where collaborative resources are available. The Reparametrized YOLOX-s model achieves a balance between performance and efficiency, with an accuracy improvement of +5.0% and a cost reduction of −16%, requiring 1100 iterations. This showcases the effectiveness of integrating lightweight architectures with precise tuning for real-time systems.

4.9.1. Comparison of Strategies

• High Iteration Techniques: Methods like Automated Hyperparameter Search (1500 iterations) and Meta-Learning Hyperparameter Tuning (1300 iterations) prioritize exhaustive exploration of parameter spaces, yielding accuracy improvements of +4.2% to +5.5%. These approaches are ideal for systems that require maximum precision and reliability.
• Moderate Iteration Techniques: Approaches such as ShuffleNet with YOLOv5 tuning (950 iterations) achieve accuracy gains of +4.8% and cost reductions of −18%, offering scalability for systems with moderate resource availability.
• Dynamic Iteration Techniques: Crowdsourced tuning strategies adapt to variable workloads, providing flexibility while maintaining strong performance.

a.

Recommendations for Optimization

To enhance model performance,

• Adopt Bayesian Optimization: This method consistently delivers reliable improvements, making it a strong choice for moderately constrained systems.
• Utilize Dynamic Tuning Approaches: Techniques such as crowdsourced optimization provide scalability for real-time or large-scale applications.
• Optimize Lightweight Architectures: Models like Reparametrized YOLOX-s demonstrate that combining lightweight designs with effective parameter tuning achieve high efficiency and accuracy.

4.9.2. Research Directions

Future efforts should focus on hybrid optimization techniques that combine Bayesian strategies with adaptive learning to improve flexibility and performance. Additionally, incorporating synthetic datasets during optimization can enhance model robustness while minimizing computational overhead. These advancements will ensure that traffic sign detection systems remain accurate and efficient in complex real-world scenarios.

4.10. Limitations in Traffic Sign and Road Marking Detection

Traffic sign and road marking detection systems face significant challenges in adapting to variations in design and standards across geographic regions. These variations include differences in shapes, colors, pictograms, and design structures, which significantly impact model performance when deployed in new regions. Addressing these discrepancies is crucial for ensuring reliable detection and recognition performance in diverse environments.

4.10.1. Geographic Variations in Traffic Signs

A study by Maletzky [185] demonstrated that models trained on German traffic sign pictograms exhibited a significant drop in accuracy when tested on Austrian pictograms. The accuracy decreased from 95% to 80%, highlighting the sensitivity of machine learning models to subtle design differences, such as contour thickness and pictogram details [185]. Similarly, variations in color standards between regions have been shown to reduce detection accuracy significantly. A study observed a decrease from 90% to 75% when models encountered signs with different hue and saturation levels [166].

4.10.2. Challenges with Road Markings

Road markings also pose challenges, particularly when models trained on North American datasets are tested on European Road markings. Differences in width, spacing, and painting materials led to a decline in detection accuracy from 92% to 70% [186]. These findings underscore the need for models to adapt to regional standards in road marking detection.

4.10.3. Dataset Limitations

A lack of diverse and representative datasets is a primary limitation for existing models. The German Traffic Sign Recognition Benchmark (GTSRB), for example, focuses exclusively on German signs. Cross-validation studies have shown that models trained on the GTSRB achieve an accuracy of 93% on German signs but only 79% on French signs [187]. Conversely, combining datasets like the GTSRB, LISA, and Mapillary improved average accuracy from 88% to 94%, emphasizing the importance of dataset diversity [188].

4.10.4. Enhanced Performance

To address these challenges, several strategies have been proposed. Incorporating traffic signs and road markings from multiple regions captures a broader range of features and improves overall model performance. Enriching datasets with traffic signs from Europe, Asia, and North America increased recognition accuracy by 7% [189]. Adopting pre-trained models to new regional contexts using transfer learning techniques has significantly improved accuracy, achieving 98% in some cases [190]. Domain Adaptation Techniques like Generative Adversarial Networks (GANs) align feature spaces between training and target regions, increasing accuracy by 15% [191].

4.11. Proposed Solutions in Traffic Sign and Road Marking Detection

Incorporating these emerging trends into a literature review would provide a more comprehensive and up-to-date perspective on embedded vision systems for detecting and recognizing traffic signs and lane markings. This would offer researchers and practitioners valuable insights into innovative approaches that can enhance road safety through more precise and adaptive detection systems.

4.11.1. Transformers Models

Embedded vision systems for detecting and recognizing traffic signs have traditionally relied on methods such as convolutional neural networks (CNNs). However, recent advancements in artificial intelligence have introduced transformer-based models, which demonstrate promising performance in this domain. For instance, the study [192] highlights how transformers outperform CNNs in highly complex environments due to their ability to process global dependencies. Furthermore, another article [193] explores the limitations of traditional approaches and introduces hybrid methodologies incorporating transformers for improved performance.

In [194], a lightweight model TSD-DETR (Traffic Sign Detection based on Real-Time Detection Transformer for Traffic Sign Detection) was proposed, shown in Figure 43. This model is applied to traffic sign detection in a complex environment successfully. The proposed model employs a structured feature extraction module that integrates various convolutional modules to capture multi-scale features at different levels, enhancing the overall feature extraction capability.

To address the challenge of detecting small traffic signs, a specialized small object detection method was introduced. The model, TSD-DETR, achieved an impressive mean average precision (mAP) of 96.8% on the Tsinghua-Tencent 100K dataset (Figure 44).

The experimental results show that TSD-DETR can detect small objects in complex environments effectively (Figure 45). It can be used in many scenarios, such as real road traffic supervision, automatic driving, and driver assistance systems.

Another paper proposed [195] the architecture below (in Figure 46), consisting of a backbone for feature extraction in a base transformer model.

The authors highlighted the efficiency of their transformer model in detecting lane lines (Figure 46), showcasing its ability to accurately fit distant lanes, multi-lane curves, and high-curvature scenes (

Table 24. Accurate fit of different scenes.

Least Squares Method	Acc	FP	FN
Quadratic curve	90.22	0.1259	0.0895
Cubic curve	93.22	0.0954	0.0715
Quartic curve	91.58	0.1061	0.0845
Quadratic curve	90.22	0.1259	0.0895

).

Their approach outperformed the Lanenet CNN model (Figure 47), achieving greater precision without drifting in challenging road scenarios.

4.11.2. Modeling Spatial Hierarchies

In parallel, new deep learning architectures are emerging as alternatives to CNNs. One article [196] demonstrated that capsule networks can effectively model complex spatial relationships and reduce errors caused by variations in the orientation and perspective of signs. The contribution of this work is the application of capsule networks (Figure 48) to detect traffic signs [197]. This paper addresses the challenges of misinterpretation caused by the use of CNNs in traffic sign recognition, particularly the limitations introduced by max pooling.

This study aimed to explore the application of capsule networks (CapsNets) in traffic sign recognition, a cutting-edge approach in deep learning for analyzing traffic environments. By fine-tuning parameters and weights, this study demonstrated how CapsNets can effectively address the challenges of traffic sign recognition. Leveraging their dynamic routing mechanism, CapsNets showed great potential in overcoming the complexities of scene interpretation in traffic scenarios. The Capsule Network achieved an average accuracy of 98.72% and an F1-score of 99.27% (Figure 49).

The developed model effectively detected color variations and accurately categorized traffic sign classes. Figure 50 presents test images alongside their prediction outcomes, with bar charts displayed on the right of each image, illustrating the probabilities of 10 classes in which the sign was correctly identified using this approach.

Additionally, this paper aimed to tackle the challenges associated with thresholding-based techniques for extracting and classifying road markings [198], which often face reduced robustness and increased computational complexity when processing high-density 3D point clouds captured by MLS systems. These issues are primarily caused by varying point density, low-intensity contrast with adjacent pavements, and irregular data structures. To address these problems, the study proposed two novel capsule network architectures, tailored specifically for the efficient extraction and classification of road markings from dense and irregular MLS point cloud data.

Figure 51 presents a U-shaped capsule network architecture designed to capture both intensity variations from large datasets of labeled image patches and the spatial and shape-related features of road markings. The network incorporates a combination of traditional convolutional layers, primary capsule layers, convolutional capsule layers, and deconvolutional capsule layers, enabling a robust feature extraction process tailored for this task.

Figure 52 and Figure 53 showcase the classification outcomes of road markings in urban and highway environments. In Figure 52, the model demonstrates a 2.16% misclassification rate, accurately identifying most markings but encountering errors with degraded lane lines, which were misclassified as dashed lines, and some zebra crossings or arrows mistakenly labeled as lane lines. Figure 53 highlights result from highway data, achieving a 4.87% misclassification rate, with similar challenges arising from broken lane lines resembling dashed lines or arrows. Despite these issues, the model performed well in classifying road markings across complex scenarios.

Figure 54 illustrates the extraction and classification of road markings from low-quality data. The processing times for data preprocessing, road marking extraction, and classification were 36.47 s, 3.07 s, and 2.58 s, respectively, with preprocessing taking the largest share. Utilizing GPU parallel computing and multi-threading techniques could substantially improve the speed of 3D point cloud projections and optimize the performance of capsule-based networks.

4.11.3. Addressing Complex Backgrounds and Enhancing Model Performance

The accurate recognition of traffic signs in real-world environments is significantly hindered by complex backgrounds containing elements such as pedestrians, vehicles, and other objects. These challenges lead to an increased rate of false positives, which directly impacts the reliability of detection systems. For example, studies have reported that in dense urban environments, false positive rates can rise by 15–20% due to the interference of background elements [199]. This underscores the need for robust algorithms capable of effectively distinguishing traffic signs from surrounding distractions, especially in cluttered scenes. To illustrate the impact of complex backgrounds on detection performance,

Table 25. Comprehensive comparison of advanced traffic sign detection models in complex real-world environments.

Model	Detection Accuracy (%)	Positive False Rate (%)	Processing Time (ms)	Robustness to Complex Backgrounds	Scalability to Larger Datasets	Hardware Dependency
YOLO-SG [200]	95.3	4.7	25	High	High	GPU Required
ESSD [201]	93.2	6.8	30	Moderate	High	GPU Required
TrafficSignNet [202]	91.5	8.5	28	Moderate	Moderate	Moderate
Color-Based Segmentation [203]	85	15	20	Low	Low	Low
HOG-Based Detection	87	12	35	Moderate	Moderate	Low
Faster R-CNN	92	7	40	High	High	GPU Required
SSD with FPN	93.8	6.2	32	High	High	GPU Required
Multi-Scale Attention Network [204]	91.2	7.5	31	High	High	GPU Required

compares key parameters across various models.

Table 25 provides a comprehensive comparison of advanced traffic sign detection models in complex real-world environments, such as urban settings with pedestrians, vehicles, and other distractions. Among these models, YOLO-SG stands out with a detection accuracy of 95.3% and a low false positive rate of 4.7%, making it highly effective for real-time applications. Its processing time of just 25 milliseconds reinforces its suitability for systems requiring rapid decision-making. SSD with an FPN also demonstrates strong performance, achieving an accuracy of 93.8% and a false positive rate of 6.2%, showcasing its robustness in diverse scenarios.

In contrast, methods like ESSD (93.2% accuracy, 6.8% false positives) and the Faster R-CNN (92.0% accuracy, 7.0% false positives) exhibit slightly higher computational demands, with processing times of 30 milliseconds and 40 milliseconds, respectively. Simpler techniques, such as HOG-Based Detection and Color-Based Segmentation, lag behind with lower accuracy of 87.0% and 85.0%, and false positive rates of 12.0% and 15.0%, reflecting their limited adaptability to complex backgrounds. The MTSDet model, which utilizes multi-scale feature extraction, achieves a respectable accuracy of 91.2% and a false positive rate of 7.5%, highlighting its potential for applications requiring scalable and efficient detection solutions.

Enhancing detection systems could involve incorporating context-aware deep learning models capable of interpreting environmental details. Hybrid approaches combining convolutional neural networks (CNNs) with attention mechanisms or transformers can better capture contextual relationships in cluttered scenes. Additionally, augmenting training datasets with synthetic data can address the lack of diversity in real-world datasets, enabling models to handle rare and challenging scenarios effectively. To further improve performance, future research should prioritize lightweight model architectures optimized for edge devices, enabling real-time deployment. Investigating semi-supervised or unsupervised learning methods could reduce the dependency on extensive labeled datasets. Additionally, refining multi-scale feature fusion techniques could enhance detection accuracy for both small and large traffic signs. By pursuing these strategies, traffic sign detection systems can achieve higher efficiency, scalability, and reliability, supporting advancements in autonomous vehicles and intelligent transportation systems.

4.12. Advancing Intelligent Transportation Through Hybrid Methodologies

The combination of different methods into hybrid approaches is a promising area that remains insufficiently explored in this context. By blending the strengths of various techniques, hybrid methods can overcome the individual limitations of each approach, resulting in improved accuracy, resilience, and adaptability to challenging environmental conditions. For example, integrating AI-driven algorithms with geometric and color-based features can provide enhanced performance in situations with poor visibility or intricate surroundings. Adding a section to examine these hybrid approaches and their advantages would significantly enhance the paper by providing deeper insights into how such combinations can push forward advancements in vision-based systems for intelligent transportation. This inclusion would also underline the importance of synergistic strategies in improving both road safety and the reliability of these systems.

The Region of Interest (ROI)

The region-of-interest (ROI) approach plays a crucial role in image processing and computer vision by focusing analysis on specific parts of an image that are likely to contain relevant information, such as traffic signs. For traffic sign detection, ROIs are often determined using preprocessing methods like color segmentation or edge detection to isolate regions with features typical of traffic signs, such as red circular outlines or triangular shapes. For instance, thresholding techniques in color spaces such as HSV or RGB can be applied using (Equation (55)):

$$\mathrm{S}\left(\mathrm{x},\mathrm{y}\right)\left\{\begin{array}{c}1, \mathrm{i}\mathrm{f} {\mathrm{T}}_{\mathrm{m}\mathrm{i}\mathrm{n}}\le \mathrm{I}\left(\mathrm{x},\mathrm{y}\right)\le {\mathrm{T}}_{\mathrm{m}\mathrm{a}\mathrm{x}}\\ 0, \mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{w}\mathrm{i}\mathrm{s}\mathrm{e},\end{array}\right.$$

(55)

where I(x, y) is the pixel intensity at a given point (x, y), and T_min and T_max define the range of intensity values corresponding to traffic sign colors. After identifying the ROI, techniques like the Hough Transform (Equation (56)) can be used to detect circular shapes in the region, modeled by the following equation:

$$(\mathrm{x}-\mathrm{a}{)}^2+(\mathrm{x}-\mathrm{b}{)}^2={\mathrm{r}}^2$$

(56)

where (a, b) represents the circle’s center, and r is its radius. For triangular shapes, contour detection combined with polygonal approximation techniques is employed to refine the search.

In more advanced AI-driven systems, ROIs are dynamically generated by region proposal networks (RPNs), integrated into deep learning models such as the Faster R-CNN. RPNs predict bounding boxes for ROIs, parameterized as (x, y, w, h), where x, y, denote the center, and w, h, the width and height. The optimization of these bounding boxes is achieved through a loss function (57) like

$${\mathrm{L}}_{\mathrm{r}\mathrm{e}\mathrm{g}}\left(\mathrm{t},{\mathrm{t}}^{\mathrm{*}}\right)=\sum _{\mathrm{i}ϵ\{\mathrm{x},\mathrm{y},\mathrm{w},\mathrm{h})}{\mathrm{s}\mathrm{m}\mathrm{o}\mathrm{o}\mathrm{t}\mathrm{h}}_{{\mathrm{L}}_1}\left({\mathrm{t}}_{\mathrm{i}}-{\mathrm{t}}_{\mathrm{i}}^{\mathrm{*}}\right)$$

(57)

where t and t* refer to the predicted and actual bounding box parameters, while smoothL1 ensures robustness against outliers. By leveraging ROI techniques, these systems not only improve detection accuracy but also reduce false positives.

The authors in [148] proposed (Figure 55) a hybrid technique designed for traffic sign detection, utilizing both color and shape characteristics unique to each sign category. By applying a region-of-interest (ROI) module to narrow the search area, the approach integrates color transformation to emphasize key features, template matching for shape identification, and postprocessing to refine the results, ensuring both speed and accuracy.

The authors designed a traffic sign recognition model leveraging a mix of HOG and color histogram features for precise classification. On the GTSDB benchmark, it achieved impressive AUC scores 100% for prohibitory signs and 98.85% for danger signs. With testing times ranging from 0.4 to 1.0 s per image, the model demonstrates strong potential for real-time applications, including autonomous driving systems (Figure 56).

Table 26 illustrates the strategic application of region-of-interest (ROI) techniques to optimize the detection and recognition processes in traffic sign and road lane boundary datasets. Static ROIs are predominantly employed in controlled environments, such as the GTSRB and Belgium Road Code datasets, where they ensure a focused analysis on predefined areas containing traffic signs, leading to high recognition accuracy (98.33%) with minimal false positives. Conversely, dynamic ROIs adapt to varying road scenarios, proving effective in real-time applications like road lane detection and datasets with continuous environmental changes, such as KITTI and video-based road datasets. Combining ROIs with advanced methods like CNNs, YOLOv8, and the Hough Transform enhances computational efficiency by narrowing down the area of interest while maintaining high detection scores (up to 98%). This dual approach demonstrates ROIs’ versatility in addressing both structured and dynamic environments, making them indispensable for real-world implementations in autonomous navigation and intelligent transportation systems.

Traffic sign recognition systems often face challenges due to varying lighting conditions, such as low light at night, direct sunlight, and shadow variations, which can reduce detection accuracy. For instance, studies have shown that complex lighting conditions can lead to omissions and inaccurate positioning during traffic sign detection [182]. Additionally, images captured at night are often underexposed, lacking essential details for accurate detection [205].To mitigate these issues, various methods have been proposed. Adaptive image enhancement algorithms can improve image quality in challenging lighting conditions, facilitating more accurate detection of traffic signs [182].

Furthermore, the application of color-based segmentation techniques, even in low-light conditions, can effectively isolate traffic signs [206]. Photometric invariants have also been explored to improve road sign segmentation, particularly in the face of illumination changes [207]. Deep neural networks, such as YOLO-based models, have shown increased robustness in traffic sign detection under various lighting conditions. For example, models like TrafficSignNet, built on the YOLOv8 architecture, have been developed to enhance traffic sign recognition for autonomous vehicles, even in the presence of lighting variations [202]. Additionally, approaches combining transfer learning and YOLO models have been explored to improve road sign detection across diverse lighting conditions [208].

Future research must address these challenges in detail, integrating image enhancement techniques, employing advanced deep learning models, and creating more diverse datasets that reflect a wide range of lighting conditions. For instance, datasets like the Mapillary Traffic Sign Dataset, which covers a variety of scenes and lighting scenarios, can contribute to improving the accuracy and reliability of traffic sign detection and classification algorithms globally [209].

The qualitative analysis in Table 26 demonstrates that hybrid methods, combining ROI strategies with modern deep learning models like CNNs, YOLOv8, or color-based segmentation, tend to perform better in both detection accuracy (95–97%) and false positive reduction (as low as 1.8%). Notably, these approaches are implemented on embedded platforms such as Jetson Xavier NX and TX2, where they achieve frame rates suitable for real-time applications (often above 20 FPS). In contrast, traditional techniques often do not specify execution times or hardware constraints and typically show higher false positive rates. These observations suggest that hybrid architectures offer a more balanced trade-off between precision, speed, and hardware feasibility, making them more appropriate for intelligent and embedded road perception systems.

Table 26. The strategic application of ROI techniques for detection and recognition processes in traffic signs and road lanes.

Category	Technique Used	Detection Score (%)	Recognition Score (%)	Dataset	Execution Time (ms)	False Positive Rate (%)	Type of ROI	Reference	Method	Hardware Platform/Real-Time Specs	FPS (Frames per Second)
Traffic Signs	RANSAC, ROI, Triangular Shapes	95	No specified	Belgium Road Code	No specified	2.5	Static	M. Boumediene [210]	Traditional	Not specified	N/S
	ROI, SVM, CNN	94	98.33	GTSRB	20	3	Static	N. Hasan [211]	Hybrid	GPU (unspecified), ~20 ms	50 FPS approx.
	ROI, YOLOv8	96	No specified	CCTSDB dataset	Real Tie	1.8	Dynamic	Y. Luo [212]	Hybrid	Jetson Xavier NX @21FPS, 15 W	21 FPS
	ROI, CNN, Color Segmentation	95	94	TTSDCE dataset	25	2.3	Static	Y. Zhu [213]	Hybrid	GPU (assumed), ~25 ms	40 FPS est.
Road Lane Boundaries	ROI, Dynamic, Modified Hough Transform	96	N/A	Road videos	Real Time	No specified	Dynamic	Y. Shen [214]	Hybrid	PC-based, estimated RT	RT (est.)
	ROI, Hough Transform	98	N/A	640 × 480 pixel video	5	1.2	Static	M. H. Syed [215]	Traditional	Desktop CPU (5 ms)	200 FPS est.
	ROI, Adaptive, Stereo Vision	97	N/A	Road dataset	No specified	No specified	Dynamic	Yingfo Chen [216]	Hybrid	Jetson TX2	15–20 FPS
	ROI, (CNN)	97	N/A	Several datasets	Real Time	No specified	Dynamic	A. Gudigar [217]	Hybrid	GPU or Jetson (N/S	30–40 FPS est.
	ROI, Segmentation	93	N/A	KITI	15	2.8	Static	S. P. Narote [114]	Traditional	Not specified	N/S

Table 26. The strategic application of ROI techniques for detection and recognition processes in traffic signs and road lanes.

Category	Technique Used	Detection Score (%)	Recognition Score (%)	Dataset	Execution Time (ms)	False Positive Rate (%)	Type of ROI	Reference	Method	Hardware Platform/Real-Time Specs	FPS (Frames per Second)
Traffic Signs	RANSAC, ROI, Triangular Shapes	95	No specified	Belgium Road Code	No specified	2.5	Static	M. Boumediene [210]	Traditional	Not specified	N/S
	ROI, SVM, CNN	94	98.33	GTSRB	20	3	Static	N. Hasan [211]	Hybrid	GPU (unspecified), ~20 ms	50 FPS approx.
	ROI, YOLOv8	96	No specified	CCTSDB dataset	Real Tie	1.8	Dynamic	Y. Luo [212]	Hybrid	Jetson Xavier NX @21FPS, 15 W	21 FPS
	ROI, CNN, Color Segmentation	95	94	TTSDCE dataset	25	2.3	Static	Y. Zhu [213]	Hybrid	GPU (assumed), ~25 ms	40 FPS est.
Road Lane Boundaries	ROI, Dynamic, Modified Hough Transform	96	N/A	Road videos	Real Time	No specified	Dynamic	Y. Shen [214]	Hybrid	PC-based, estimated RT	RT (est.)
	ROI, Hough Transform	98	N/A	640 × 480 pixel video	5	1.2	Static	M. H. Syed [215]	Traditional	Desktop CPU (5 ms)	200 FPS est.
	ROI, Adaptive, Stereo Vision	97	N/A	Road dataset	No specified	No specified	Dynamic	Yingfo Chen [216]	Hybrid	Jetson TX2	15–20 FPS
	ROI, (CNN)	97	N/A	Several datasets	Real Time	No specified	Dynamic	A. Gudigar [217]	Hybrid	GPU or Jetson (N/S	30–40 FPS est.
	ROI, Segmentation	93	N/A	KITI	15	2.8	Static	S. P. Narote [114]	Traditional	Not specified	N/S

4.13. Analysis of Histogram Equalization and CLAHE Techniques for Traffic Sign and Lane Detection

Histogram Equalization (HE), mentioned in several articles and shown in Table 26, is a useful technique for globally enhancing image contrast by redistributing pixel intensity levels. However, its limitations include excessive noise amplification in low-light environments, which reduces its effectiveness in advanced applications. For example, in [218], HE alone achieved a 20% contrast improvement, but its impact on detection rates remained limited compared to CLAHE.

For Histogram Equalization, the transformation of pixel intensity levels is calculated by redistributing intensities cumulatively. The key Equation (58) used is

$${\mathrm{s}}_{\mathrm{k}}=\mathrm{T}\left({\mathrm{r}}_{\mathrm{k}}\right)=\left(\mathrm{L}-1\right)\sum _{\mathrm{j}=0}^{\mathrm{k}}{\displaystyle \frac{\mathrm{p}\left({\mathrm{r}}_{\mathrm{j}}\right)}{\mathrm{N}}}$$

(58)

where

• $s_k$ is the new normalized intensity;
• $r_k$ is the original pixel intensity;
• L is the total number of intensity levels (typically 256 for an 8-bit image);
• $p\left(r_j\right)$ is the probability of intensity rjr_jrj;
• N is the total number of pixels.

In paper [219], combining HE with CLAHE improved mAP metrics by 15%, confirming HE’s usefulness as a preliminary step, though its standalone performance is insufficient for complex systems. For CLAHE, this approach adapts contrast locally. CLAHE divides the image into small regions called “tiles,” applies histogram equalization to each tile, and limits contrast amplification by clipping intensities beyond a threshold (clip limit). The redistribution of levels is given by the following function (59):

$${\mathrm{T}}_{\mathrm{C}\mathrm{L}\mathrm{A}\mathrm{H}\mathrm{E}}\left({\mathrm{r}}_{\mathrm{k}}\right)=\min \left(\mathrm{T}\left({\mathrm{r}}_{\mathrm{k}}\right),\mathrm{c}\mathrm{l}\mathrm{i}{\mathrm{p}}_{\mathrm{l}\mathrm{i}\mathrm{m}\mathrm{i}\mathrm{t}}\right)$$

(59)

where

• $T\left(r_k\right)$ is the HE transformation applied locally;
• $cli{p}_{limit}$ is the threshold controlling the maximum amplification

These equations enable the calculation of new pixel intensities to enhance image quality, making them more effective for use in advanced vision models. On the other hand, Contrast-Limited Adaptive Histogram Equalization (CLAHE) stands out for its ability to enhance local contrast while minimizing artifacts, making it a key solution for low-light environments. The articles in Table 27 confirm its superiority: in [220], CLAHE combined with ResNet achieved a 95.7% classification accuracy, and in article #8, integrating CLAHE with YOLOv5 reached a 95.2% detection rate. CLAHE also improves recognition and detection performance of traffic signs in complex nighttime environments, as demonstrated in [221], with a 12% increase in F1-score. These results highlight CLAHE as the preferred technique for advanced vision systems requiring high precision in challenging conditions.

Figure 57 demonstrates the effect of CLAHE on a low-light driving scene [222]. Compared to the original image, the CLAHE-enhanced version shows significantly improved local contrast, making lane markings and surrounding structures more visible. This enhancement facilitates better feature extraction and improves the reliability of detection algorithms under poor lighting conditions, highlighting CLAHE’s suitability for real-time vision systems in nighttime environments. The authors presented annotated images comparing the original state with the result after applying CLAHE, clearly illustrating visible improvements in brightness and contrast (with a mAP of 76.2%).

Applying CLAHE (Figure 58) [223] as a preprocessing step significantly enhances image quality under low-light conditions. Compared to other enhancement methods such as Color Stretching and standard Histogram Equalization, CLAHE offers clearer contrast and preserves fine details without amplifying noise. This improvement makes critical features like road signs and lane markings more distinguishable, which in turn strengthens the performance of YOLO-based detection models. The resulting output and evaluation curves clearly reflect this benefit, highlighting CLAHE’s effectiveness in improving detection reliability in nighttime or poorly lit driving environments.

Table 27. Comparison of Histogram Equalization and CLAHE in Traffic Sign and Road Marking Recognition Studies.

Authors	Year	Techniques Used	Histogram Equalization	CLAHE	Application	Results/Key Values
Utkarsh Dubey, Rahul Kumar Chaurasiya [220]	2021	CLAHE, CNN (ResNet)	Not Used	Used	Traffic sign recognition	Classification accuracy: 95.7% after CLAHE
Jiana Yao et al. [221]	2023	Histogram Equalization, CLAHE, Mask RCNN	Used	Used	Detection and recognition of traffic signs	12% increase in F1-Score under low-light conditions
Chen [223]	2024	CLAHE, YOLO	Not Used	Used	Recognition of road markings	Detection rate improved to 91% under low-light conditions
Manongga [224]	2024	CLAHE, Fusion Linear Image Enhancement, YOLOv7	Not Used	Used	Detection of markings	Detection rate: 89.6%; reduction in false positives by 15%
Yan [165]	2023	CLAHE, Lightweight Model	Not Used	Used	Traffic sign recognition	Accuracy improvement of 8% compared to non-enhanced images
Wang [225]	2023	Histogram Equalization, CLAHE	Used	Used	Detection of signs	Performance improvement to 93.2% in nighttime scenarios
Shuen Zhao et al. [226].	2024	CLAHE, Correction Gamma, CNN	Not Used	Used	Signs and markings	Accuracy: 94.8%; reduction in inference time by 25%
Sun [227]	2024	CLAHE, YOLOv5	Not Used	Used	Detection of signs	Detection rate: 95.2%; increase in detection speed by 20%
Prasanthi [219]	2022	Histogram Equalization, CLAHE, CNN	Used	Used	Lanes and signs	15% improvement in mAP metrics under low-light conditions
Qin [218]	2019	Histogram Equalization	Used	Not Used	Improvement in sign image	20% contrast improvement; reduction in detection errors

Table 27. Comparison of Histogram Equalization and CLAHE in Traffic Sign and Road Marking Recognition Studies.

Authors	Year	Techniques Used	Histogram Equalization	CLAHE	Application	Results/Key Values
Utkarsh Dubey, Rahul Kumar Chaurasiya [220]	2021	CLAHE, CNN (ResNet)	Not Used	Used	Traffic sign recognition	Classification accuracy: 95.7% after CLAHE
Jiana Yao et al. [221]	2023	Histogram Equalization, CLAHE, Mask RCNN	Used	Used	Detection and recognition of traffic signs	12% increase in F1-Score under low-light conditions
Chen [223]	2024	CLAHE, YOLO	Not Used	Used	Recognition of road markings	Detection rate improved to 91% under low-light conditions
Manongga [224]	2024	CLAHE, Fusion Linear Image Enhancement, YOLOv7	Not Used	Used	Detection of markings	Detection rate: 89.6%; reduction in false positives by 15%
Yan [165]	2023	CLAHE, Lightweight Model	Not Used	Used	Traffic sign recognition	Accuracy improvement of 8% compared to non-enhanced images
Wang [225]	2023	Histogram Equalization, CLAHE	Used	Used	Detection of signs	Performance improvement to 93.2% in nighttime scenarios
Shuen Zhao et al. [226].	2024	CLAHE, Correction Gamma, CNN	Not Used	Used	Signs and markings	Accuracy: 94.8%; reduction in inference time by 25%
Sun [227]	2024	CLAHE, YOLOv5	Not Used	Used	Detection of signs	Detection rate: 95.2%; increase in detection speed by 20%
Prasanthi [219]	2022	Histogram Equalization, CLAHE, CNN	Used	Used	Lanes and signs	15% improvement in mAP metrics under low-light conditions
Qin [218]	2019	Histogram Equalization	Used	Not Used	Improvement in sign image	20% contrast improvement; reduction in detection errors

4.14. Board Experimentation for Detection and Recognition

4.14.1. Traffic Signs

The results in Figure 59 in [228] for detection and recognition traffic signs reveal that the combination (SSD + MobileNetv1 FPN) on TPU emerged as the most effective. This configuration consistently outperformed others across all stages and nearly all metrics. Notably, it achieved a training speedup of 16.3 and maintained a minimal accuracy difference of 0.0363 between CPU and GPU setups (Figure 59).

The research presented in [229] introduced a k-means clustering function and demonstrated its application in traffic sign recognition, achieving significant speed improvements compared to software execution on an ARM CPU. The study reported impressive maximum speedups of eight times for k-means clustering and 9.6 times for traffic sign recognition. Notably, the implementation utilizing 16 IPPro cores exhibited remarkable power efficiency, being 57 times, 28 times, and 1.7 times more power-efficient (measured as frames per second per watt, fps/W), respectively, as illustrated in

Table 28. Acceleration of color and morphology operations using IPPro on Zedboard.

Description	Color	Morphology
No. of cores	32	1,643,588 (41%)
FF	41,624 (39%)	33,545 (63%)
LUT	29,945 (56%)	48 (22%)
BRAM	60 (42%)	112 (80%)
Cycles/Pixel	160	26
Exec. (ms)	19.7 (8.7 *)	41.3 (18.3 *)
Speed-up	4.5 (10.3 *)	9.6 (21.75 *)

* Achievable performance using Zynq XC7Z045 Kintex-7.

, than the ARM Cortex-A7 CPU, NVIDIA GeForce GTX980 GPU, and ARM Mali-T628 embedded GPUs.

4.14.2. Road Lines

A refined implementation [230] of the Hough Transform (HT) on FPGA was introduced for straight-lane line detection. The optimization involves filtering horizontal lines to simplify the mapping equation of HT. Additionally, filtering vertical lines and defining two regions of interest (ROIs) for edge gradient orientation further enable the HT to consumes less than 0.1% of the available FPGA memory [231,232]. Moreover, this architecture facilitates the inverse mapping operation from Hough Space to the image plane without incurring any additional hardware costs (Figure 60).

The model discussed in [233] achieved an accuracy of 93.53% on the TuSimple dataset, running at 348.34 frames per second (FPS) on the NVIDIA Tesla T4 GPU with an image size of 256 × 512. To optimize processing speed and minimize power consumption, a hardware accelerator was implemented on the Virtex-7VC707 FPGA [234]. Through optimization techniques such as data quantization and dual 8-bit multiplications on a single multiplier, the hardware accelerator achieved impressive performance, operating at 640 FPS at 250 MHz while consuming only 10.309 W. This resulted in outstanding system throughput and energy efficiency, measured at 345.6 GOPS and 33.52 GOPS/W, respectively (as illustrated in Figure 61).

The accompanying

Table 29. Reported precisions for traffic sign and lane detection using various hardware and datasets.

	Authors	Board	Datasets	Precision (%)
Traffic Signs	F. Zaklouta [235]	Raspberry Pi	GTSRB	90
	N. Triki [236]	Raspberry Pi	GTSRB	98.56
	Yan Han [237]	FPGA	U.S. traffic signs	95
	W. Farhat [238]	Zynq FPGA	Original	97.72
	R. Hmida [239]	Virtex-5 FPGA	Tunisian and European road signs	91
Road Lane	S. Malmir [240]	XC7K325T2FFG900C FPGA	Caltech datasets and KITTI dataset	97.80
	T.Yau Teo [241]	Raspberry	Malaysia roads	95.24
	H. Gajjar [242]	Raspberry Pico and NVIDIA Jetson Nano	The data from the depth camera perceived using OpenNI library	98.50
	J. Suder [243]	Raspberry Pi 4 model B and NVIDIA Jetson Nano	CuLane dataset	97
	Y. Guo [244]	NVIDIA TITAN XP GPU	GNSS data	99.4

provides a synthesis of research efforts focused on traffic sign and lane detection methods. Key considerations include the hardware platforms utilized, datasets employed, and the corresponding accuracy levels achieved in each study.

The hardware used by the authors includes Raspberry Pi, FPGA, and NVIDIA Jetson Nano. The datasets used include GTSRB, U.S. traffic signs, Tunisian road signs, Caltech datasets, KITTI dataset, Malaysian roads, and CuLane dataset. The accuracy achieved by the authors varies depending on the hardware, dataset, and method used.

The authors used a variety of methods to detect traffic signs and lanes. These methods include deep learning, machine learning, and computer vision. Deep learning methods are based on artificial neural networks that can learn to recognize patterns in data. Machine learning methods are based on statistical models that can be used to predict outcomes. Computer vision methods are based on algorithms that can be used to extract information from images.

The authors found that deep learning methods generally achieve the highest accuracy. However, deep learning approaches necessitate substantial amounts of data and computational resources. Machine learning methods are less accurate than deep learning methods, but they require less data and computational resources. Computer vision methods are the least accurate, but they require the least amount of data and computational resources.

The authors’ work is significant because it provides a comprehensive overview of traffic sign and lane detection methods. The authors’ work can be used to guide future research in this area.

5. Research Gaps and Challenges

While substantial advancements have been achieved in embedded vision systems for traffic sign and lane recognition, several critical challenges remain:

• Dataset Limitations

Most existing datasets lack sufficient diversity in lighting, weather, and road conditions. This undermines model generalizations and reliability in real-world deployments.

• Real-Time Performance on Embedded Platforms

Many deep learning models demonstrate excellent accuracy but fail to meet latency and memory constraints when deployed on low-power embedded hardware.

• Sensor Fusion Complexity

While combining data from LiDAR, radar, and cameras can improve robustness, there is limited work on efficient fusion strategies that maintain real-time performance in embedded systems.

• Explainability and Trustworthiness

The integration of explainable AI (XAI) is underexplored in this domain. Achieving interpretable yet performant systems remains a key challenge, particularly for safety-critical applications.

• Lack of Standard Benchmarks:

The absence of unified benchmarking protocols for embedded ADASs prevents consistent evaluation and comparison across models and datasets.

Addressing these research gaps is essential to ensure the scalability, transparency, and reliability of embedded vision systems in the evolving landscape of intelligent transportation.

6. Conclusions

This review has provided a structured and comprehensive synthesis of embedded vision systems for traffic sign and lane detection in the automotive sector. By categorizing and evaluating state-of-the-art approaches through a SALSA-based methodology, we highlighted both technological advances and unresolved challenges. The inclusion of real-time constraints, sensor limitations, and deep learning models offers a pragmatic view of current capabilities and future potential. Specifically, this study introduces a practical taxonomy of recognition methods, structured summary tables, and an extended set of research questions including the role of embedded processors, all aimed at clarifying the trade-offs between accuracy, computational cost, and real-time feasibility. We hope this work serves as a valuable reference for researchers and engineers working on safer and smarter transportation systems. This study’s implications extend beyond the automotive sector, offering insights into the broader field of embedded vision systems and their role in advancing intelligent transportation systems.

7. Future Research Directions

Although this review has provided a comprehensive overview of current traffic sign and lane marking recognition systems, several specific areas merit further investigation. One important direction involves the development of adaptive algorithms capable of maintaining high performance under extreme conditions, such as occlusion, glare, and low visibility. A relevant research question could be the following: how can Vision Transformer models be optimized for deployment on real-time edge computing platforms without compromising accuracy or latency? Another promising area is the fusion of multimodal sensory data. For instance, investigating fusion strategies that combine LiDAR and vision data—particularly under occlusion-heavy scenarios—could yield more robust detection pipelines. Comparative experiments could assess early versus late fusion architectures across varied environmental datasets. The integration of explainable AI (XAI) is also an essential research avenue. Identifying which XAI techniques provide the best balance between interpretability and inference speed in embedded contexts would support the development of safer and more transparent systems.

Finally, the lack of standardized benchmarking tools remains a barrier. Future work should prioritize the creation of evaluation protocols and datasets tailored to embedded vision systems, enabling reproducible and comparable assessments across architectures. These targeted directions are critical to advancing the scalability, reliability, and transparency of vision-based perception in autonomous driving technologies.