Survey on Image-Based Vehicle Detection Methods

Abstract

Vehicle detection is essential for real-world applications such as road surveillance, intelligent transportation systems, and autonomous driving, where high accuracy and real-time performance are critical. However, achieving robust detection remains challenging due to scene complexity, occlusion, scale variation, and varying lighting conditions. Over the past two decades, numerous studies have been proposed to address these issues. This study presents a comprehensive and structured survey of image-based vehicle detection methods, systematically comparing classical machine learning techniques based on handcrafted features with modern deep learning approaches. Deep learning methods are categorized into one-stage detectors (e.g., YOLO, SSD, FCOS, CenterNet), two-stage detectors (e.g., Faster R-CNN, Mask R-CNN), transformer-based detectors (e.g., DETR, Swin Transformer), and GAN-based methods, highlighting architectural trade-offs concerning speed, accuracy, and practical deployment. We analyze widely adopted performance metrics from recent studies, evaluate characteristics and limitations of popular vehicle detection datasets, and explicitly discuss technical challenges, including domain generalization, environmental variability, computational constraints, and annotation quality. The survey concludes by clearly identifying open research challenges and promising future directions, such as efficient edge deployment strategies, multimodal data fusion, transformer-based enhancements, and integration with Vehicle-to-Everything (V2X) communication systems.

1. Introduction

Real-world applications in domains such as automated parking management, traffic monitoring, and intelligent transportation systems (ITS) utilize vehicle detection to enhance operational efficiency and road safety. These technologies optimize traffic flow and support effective transport system management [1,2,3]. The need for reliable vehicle detection has grown due to increasing traffic congestion, rising accident rates, and the widespread deployment of surveillance infrastructure such as road-mounted cameras [4]. In addition to enabling real-time traffic analysis, vehicle detection supports public safety functions, including detecting traffic violations such as speeding, unauthorized vehicle movement, red-light infractions, and helmet non-compliance [5]. Beyond ITS, the rapid development of autonomous driving technologies has further intensified the demand for high-performance vehicle detection systems. These systems serve as critical components for perception and decision-making pipelines, enabling vehicles to accurately interpret dynamic road environments. For instance, integrated frameworks that combine vehicle detection with deep reinforcement learning have shown significant improvements in high-speed cruising safety and responsiveness in autonomous navigation [6]. Similarly, real-time detection models such as YOLOv4-5D and YOLO-EfficientDet hybrids have demonstrated improved accuracy and computational efficiency for real-world deployment in autonomous vehicles [7,8]. Dedicated solutions for tasks like traffic sign detection and general object localization have further contributed to reliable autonomous driving behavior [9,10].

While existing reviews have broadly addressed object detection and traffic surveillance, this survey adopts a more application-oriented perspective by categorizing vehicle detection methods according to their suitability for real-time operation, edge deployment, and integration within autonomous driving pipelines. It unifies classical and modern deep learning approaches, emphasizing speed, accuracy, and hardware efficiency trade-offs, and it highlights their relevance to emerging paradigms such as cooperative V2X systems and edge–cloud architectures. Recent research by Liang et al. [11] proposed a MAS-based hierarchical control architecture for connected and automated vehicles, underscoring the importance of accurate vehicle detection as a foundational layer for cooperative perception and decision-making. Special attention is given to the evolution of lightweight, high-throughput detection frameworks, including one-stage detectors, Transformer-based attention models, and recent architectural innovations. The survey also synthesizes evaluation metrics, dataset characteristics, and practical benchmarks to support informed model selection across diverse deployment scenarios.

An overview of the general workflow for image-based vehicle detection is shown in Figure 1, which outlines the key stages from dataset preprocessing to model training and evaluation. This logical framework contextualizes the typical operations performed during detection model development. In practice, a vehicle detection system processes visual input from onboard or roadside sensors to identify and classify vehicles. Figure 2 illustrates this process, where an object detection model takes a raw image and produces structured detections comprising bounding boxes, associated class labels, and confidence scores for detected vehicles. This output supports downstream applications such as traffic monitoring and autonomous decision-making. Recent research has introduced attention mechanisms, optimized deep learning architectures, and edge-based inference systems to improve detection performance under real-world conditions [12,13,14,15]. The growing complexity of traffic scenes—due to occlusion, varying vehicle sizes, diverse shapes, and lighting conditions—has made vision-based detection a central focus of intelligent transportation research. Camera-based detection is particularly popular among sensor modalities due to its cost-effectiveness and high-resolution data [16]. Convolutional Neural Networks (CNNs) and other deep learning models are widely adopted for their ability to learn robust visual features directly from raw input, thus offering significant advantages over traditional handcrafted methods. Recent studies have explored hybrid object detection architectures to combine the strengths of two-stage and one-stage detectors, aiming to balance detection accuracy and inference speed. Soviany et al. [17] proposed an optimization framework that systematically tunes trade-offs between single-stage and two-stage detectors. Moreover, the emergence of RT-DETR [18,19] demonstrates that end-to-end Transformer-based models with efficient attention can rival traditional detectors in real-time performance. These models utilize sparse query-based detection mechanisms that simplify post-processing and enhance scalability. Additionally, combining YOLOv4 with EfficientDet has been shown to boost both precision and inference speed for autonomous vehicle applications [8]. Future research could investigate such hybrid paradigms, including prompt-based Transformers and adaptive attention modules, to develop scalable models suitable for edge deployment in dynamic environments.

Earlier vehicle detection methods relied on classical machine learning techniques such as Scale-Invariant Feature Transform (SIFT), Haar-like features [20], and Histogram of Oriented Gradients (HOG), which require manual feature engineering [21]. Although effective in some scenarios, these methods generally suffer from high computational overhead and lack scalability for real-time use. Deep learning models, in contrast, can automatically learn multi-level features from large datasets, thereby improving accuracy and reducing the need for manual intervention [22].

Deep learning-based vehicle detection methods can be broadly categorized into one-stage and two-stage detectors. One-stage models such as the YOLO (You Only Look Once) series—from YOLOv1 to YOLOv12—directly predict bounding boxes and class labels in a single pass, making them ideal for real-time applications [23,24,25,26,27]. Recent anchor-free one-stage models such as FCOS and CenterNet have emerged to simplify detection pipelines and improve performance under occlusion and scale variation. In contrast, two-stage detectors such as R-CNN, Fast R-CNN, and Faster R-CNN first generate region proposals and then perform classification and refinement [28,29,30], typically achieving higher accuracy but at the cost of increased computation. This paper comprehensively reviews vehicle detection methods, tracing the evolution from classical machine learning techniques to modern deep learning architectures. It also surveys benchmark datasets, evaluation metrics (e.g., mAP, IoU, FPS), and recent performance results across multiple scenarios. Furthermore, the paper discusses key challenges in real-world deployment, including scene complexity, environmental variability, and edge-device limitations. It concludes with future directions such as V2X integration, multimodal fusion, and model generalization strategies.

2. Vehicle Detection Methods

The task of identifying the location and category of the vehicle within an input image or video is vehicle detection (see Figure 2); this task is essential for numerous intelligent transportation applications, including traffic monitoring, driver assistance technologies, parking management, and autonomous driving systems. This section discusses the methods used in the existing literature for vehicle detection. The following sections provide details on the existing approaches.

2.1. Classical Vehicle Detection Methods

Classical vehicle detection methods, widely adopted prior to the deep learning era, are based on traditional machine learning pipelines that involve handcrafted feature extraction followed by classification. While these approaches were effective in early research, they have been largely supplanted by deep learning methods due to their limited scalability, poor generalization in complex environments, and high sensitivity to noise and occlusion. Figure 3 illustrates the general processing pipeline employed by classical vehicle detection systems.

2.1.1. Feature Extraction Techniques

• Haar-like Features
  Haar-like features are simple rectangular features used to detect edges, lines, and other visual patterns by computing the difference in pixel intensities between adjacent regions [31]. These features are particularly effective for capturing structural cues in vehicle shapes. The Haar-like feature F is calculated as follows:

  $$\begin{array}{cc}\hfill F& =S_W-S_B\hfill \end{array}$$

  (1)

  $$\begin{array}{cc}\hfill S_W& =\sum _{(x,y)\in W}I(x,y),S_B=\sum _{(x,y)\in B}I(x,y)\hfill \end{array}$$

  (2)

  where $I(x,y)$ denotes the pixel intensity at coordinate $(x,y)$, and W and B represent the white and black rectangular regions, respectively. These features are evaluated across different scales and positions using an integral image for efficient computation. They are often combined with classifiers such as AdaBoost in early vehicle detection systems. Figure 4 illustrates typical Haar-like patterns used in vehicle detection tasks.
• HOG Feature Extraction
  In 2005, Dalal and Triggs [32] introduced the Histogram of Oriented Gradients (HOG), a descriptor that captures local object appearance and shape by encoding the distribution of intensity gradients or edge directions. HOG is particularly effective for detecting rigid objects such as vehicles, where edge and contour structures are prominent. The image is divided into small cells, and each cell’s histogram bins gradient orientations into predefined angle ranges (e.g., 0–180°), weighted by gradient magnitude. First, gradient magnitude and orientation are computed as follows:

  $$M=\sqrt{G_x^2+G_y^2},\theta =arctan\left({\displaystyle \frac{G_y}{G_x}}\right)$$

  (3)

  where $G_x$ and $G_y$ are the horizontal and vertical gradients, respectively.
  The image is then divided into cells, and orientation histograms are computed for each. These histograms are normalized over larger spatial blocks to improve invariance to illumination and contrast. The block normalization of the feature vector v is using L2-norm, is defined as follows:

  $$v_n={\displaystyle \frac{v}{\parallel v\parallel }}$$

  (4)
  HOG features are widely used in vehicle detection systems due to their robustness in capturing structural information while being relatively efficient to compute.
• LBP Feature Extraction
  Local Binary Patterns (LBP) encode local texture patterns by comparing each pixel to its surrounding neighbors, making it robust to lighting variations. However, it is generally less effective in scenes with complex backgrounds or non-uniform textures [33].
  The LBP descriptor is defined as follows:

  $$LB{P}_{P,R}=\sum _{p=0}^{P-1}s(i_p-i_c)\times 2^p$$

  (5)

  where $i_c$ is the intensity of the center pixel, $i_p$ denotes the intensity of the $p^{th}$ neighboring pixel, P is the number of sampling points, and R is the radius of the neighborhood. The thresholding function $s\left(x\right)$ is given by the following:

  $$s\left(x\right)=\left\{\begin{array}{cc}1\hfill & \mathrm{if}x\ge 0,\hfill \\ 0\hfill & \mathrm{if}x<0\hfill \end{array}\right.$$

  (6)
  The resulting binary values are combined into a single integer, forming a compact descriptor that captures local texture information. LBP is computationally efficient and suitable for real-time vehicle detection in well-lit environments.
• SIFT Detection
  SIFT extracts local features from input images, ensuring invariance to scale and orientation. It identifies key points, such as edges and corners, using a Difference-of-Gaussian (DoG) filter across multiple scales. It then refines these key points and computes descriptors based on gradient orientation histograms within their neighborhoods [34]. It is a robust feature extraction method used to identify distinctive keypoints that remain invariant to scale, rotation, and illumination changes. The initial step involves computing the Difference of Gaussians (DoG) to detect potential keypoints:

  $$\mathrm{DoG}(x,y,\sigma )=\left(G(x,y,k\sigma )-G(x,y,\sigma )\right)\times I(x,y)$$

  (7)

  where $G(x,y,\sigma )$ denotes a Gaussian kernel with standard deviation $\sigma$, k is a scale multiplier, and $I(x,y)$ is the input image. Following the DoG computation, SIFT performs keypoint localization, assigns orientations based on local gradient distributions, and generates a descriptor vector that encodes the spatial structure of gradients around each keypoint. These descriptors make SIFT particularly effective for detecting vehicles across varying scales, perspectives, and lighting conditions. While SIFT is computationally intensive, it remains influential for scale-invariant detection tasks.

2.1.2. Classical Classification Algorithms

• Support Vector Machine (SVM)
  The core concept of SVM is to identify an optimal hyperplane in a multidimensional feature space that best separates data points belonging to different classes [35]. This hyperplane serves as a decision boundary, partitioning the feature space into regions predominantly associated with one class or another. In vehicle detection, SVM is commonly trained to distinguish between vehicles (positive class) and non-vehicles (negative class).
  The optimal hyperplane is chosen to maximize the margin—the distance between the hyperplane and the nearest data points from each class (support vectors). When data are not linearly separable, SVM applies the kernel trick to map the input data into a higher-dimensional space where linear separation becomes feasible [36]. This enables SVM to handle non-linear classification tasks effectively. Variants of SVM include LSVM (Least Squares SVM), NLSVM (Non-Linear SVM), SSVM (Structural SVM), and NSVM (Normalized SVM), each offering specific adaptations for different data characteristics. A linear decision function can be expressed as follows:

  $$f\left(I\right)=w\times I+b$$

  (8)

  where w is the weight vector, I is the input feature vector, and b is the bias term.
• Adaptive Boosting (AdaBoost)
  AdaBoost is a powerful ensemble learning algorithm widely used in object detection, image classification, and other pattern recognition tasks [37]. The key idea is to combine multiple weak classifiers—each performing only slightly better than random guessing—into a single strong classifier with high accuracy. AdaBoost operates in a sequential manner, training each weak learner on the weighted version of the dataset. After each iteration, the weights of the misclassified examples are increased, forcing the next weak learner to focus more on the harder examples [38]. Correctly classified samples are assigned lower weights, reducing their influence in subsequent iterations. This adaptive re-weighting process ensures that the final ensemble is more focused on the difficult cases. Even if individual weak classifiers perform poorly, the combined model weighted by the accuracy of each learner can achieve excellent performance [39]. The sample weight update rule is the following:

  $$D_{t+1}\left(i\right)={\displaystyle \frac{D_t\left(i\right)exp(-{\alpha }_t{y}_i{h}_t\left(x_i\right))}{Z_t}}$$

  (9)

  where $D_t\left(i\right)$ is the weight of sample i at iteration t, $y_i$ is the true label, $h_t\left(x_i\right)$ is the prediction from the weak learner, and $Z_t$ is a normalization factor.
  Each weak learner is assigned a weight based on its classification error:

  $${\alpha }_t={\displaystyle \frac{1}{2}}ln\left({\displaystyle \frac{1-{\epsilon }_t}{{\epsilon }_t}}\right)$$

  (10)

  where ${\epsilon }_t$ is the error rate of the $t^{th}$ weak classifier.
  The final strong classifier is defined as follows:

  $$H\left(x\right)=\mathrm{sign}\left(\sum _{t=1}^T{\alpha }_t{h}_t\left(x\right)\right)$$

  (11)

  where $H\left(x\right)$ is the final strong classifier, T is the total number of weak classifiers, ${\alpha }_t$ is the weight assigned to the $t^{\mathrm{th}}$ weak classifier based on its accuracy, $h_t\left(x\right)$ is the prediction of the $t^{\mathrm{th}}$ weak classifier, and $\mathrm{sign}(·)$ is the sign function that returns $+1$ if the argument is positive and $-1$ otherwise.

Table 1. Summary of classical vehicle detection techniques. “Performance” refers to the primary evaluation metric (e.g., accuracy, precision, or mAP) reported in each respective study.

References	Methods	Tasks Performed	Performance
[40]	SIFT and SVM	Vehicle detection and recognition	65%
[41]	Haar-like features and AdaBoost	Vehicle detection and recognition	97%
[42]	Background subtraction, shadow removal, pixel analysis	Vehicle detection and counting	95%
[43]	Boosting HOG features, SVM	Detect moving vehicles	90%
[44]	Viola-Jones (V-J) and HOG + SVM	Vehicle detection	88.5%
[45]	LSVM + HOG	Multi-vehicle detection and tracking	97%
[46]	Hough transform algorithm, Haar-like features	Detection and labeling of multiple vehicles	95%
[47]	Background subtraction and frame difference	Moving vehicle detection	95.5%
[48]	Feature extraction and classification for rear-view vehicle detection	Vehicle detection	94.8%

and Table 2 show some studies that used classical vehicle detection techniques. It gives information about feature extraction and classifiers involved and highlights the performance achieved by each of the systems proposed.

2.1.3. Motion-Based Vehicle Detection

Motion-based vehicle detection focuses on the temporal dynamics of objects in video sequences, using changes in pixel intensity or motion vectors to identify vehicles. Unlike appearance-based methods that rely on static features such as edges, color, or texture, motion-based approaches leverage the relative movement between the camera and vehicles to segment and track objects of interest. These techniques are particularly useful in environments where appearance features are unreliable due to occlusion, poor lighting, or low resolution.

• Techniques and Algorithms
  Several classical algorithms implement motion-based detection. Lefaix et al. [49] proposed a system that models dominant image motion, typically resulting from the ego-motion of a moving camera, and detects vehicles as motion outliers. This method also enables time-to-collision (TTC) estimation by analyzing the divergence of flow vectors. Some recent approaches explore the use of optical flow and epipolar geometry constraints to enhance motion sensitivity for detecting overtaking vehicles or lane changes. Techmer [50] introduced a real-time segmentation algorithm that tracks motion along pre-defined lane contours. This method simplifies the segmentation task by focusing only on relevant motion direction, making it both efficient and resistant to lighting variation.
• Applications and Advantages
  Motion-based techniques are widely used in:

  -

  Speed estimation and violation detection in traffic surveillance [51].

  -

  Lightweight embedded systems with limited computational capacity.

  -

  Challenging environments such as nighttime driving or adverse weather conditions.

  These methods typically require fewer computational resources and are more resilient to occlusion and appearance variation than purely appearance-based approaches. For example, Jazayeri et al. [52] applied a Hidden Markov Model (HMM) to classify motion traces over time, achieving reliable vehicle detection in complex, cluttered scenes.
• Challenges Despite their advantages, motion-based approaches have several limitations:

  -

  Susceptibility to false positives caused by camera shake or ego-motion instability.

  -

  Difficulty separating multiple vehicles moving in the same direction or at similar speeds.

  -

  Ineffectiveness in detecting stationary or parked vehicles.

Mitigating these issues often involve incorporating robust background modeling, such as Mixture of Gaussians (MOG), or using contour-based tracking to stabilize the segmentation [51]. Deep learning models have shown superior performance by learning hierarchical features directly from raw data, eliminating the need for manual feature design. Techniques such as CNNs and Transformers not only improve precision but also enhance robustness in diverse and complex environments. Therefore, while classical approaches are valuable for understanding the historical evolution of vehicle detection, they are no longer competitive for modern real-time applications.

Table 2. Categorization of classical vehicle detection methods by feature type and technique. Appearance-based methods focus on object texture and shape; motion-based methods rely on temporal changes; hybrid approaches combine multiple cues or heuristic rules.

Category	Representative Techniques	References
Appearance-Based	HOG, LBP, SIFT, Haar + AdaBoost, Viola-Jones	[40,41,43,44]
Motion-Based	Background subtraction, Optical flow, Contour tracking, HMM	[42,47,49,50,52]
Hybrid/Heuristic	Pixel-based analysis, Hough + Haar-like, Shadow filtering	[46,48]

Table 2. Categorization of classical vehicle detection methods by feature type and technique. Appearance-based methods focus on object texture and shape; motion-based methods rely on temporal changes; hybrid approaches combine multiple cues or heuristic rules.

Category	Representative Techniques	References
Appearance-Based	HOG, LBP, SIFT, Haar + AdaBoost, Viola-Jones	[40,41,43,44]
Motion-Based	Background subtraction, Optical flow, Contour tracking, HMM	[42,47,49,50,52]
Hybrid/Heuristic	Pixel-based analysis, Hough + Haar-like, Shadow filtering	[46,48]

2.1.4. Summary of Classical Vehicle Detection Methods

Classical vehicle detection methods are based on sequential pipelines that combine handcrafted feature extraction with traditional machine learning classifiers. These approaches are typically grouped into three categories: appearance-based techniques (e.g., HOG, LBP, SIFT), motion-based methods (e.g., background subtraction, optical flow), and heuristic or hybrid strategies that incorporate rule-based logic. While these methods laid the groundwork for automated vehicle detection, they suffer from several limitations in real-world applications. Their performance is often hindered by sensitivity to lighting changes, occlusion, and background complexity. Furthermore, the reliance on manually designed features results in limited adaptability to variations in vehicle size, orientation, and environmental conditions. In terms of computational efficiency and detection accuracy, classical approaches generally fall short compared to modern alternatives. Their constrained scalability and slower inference times make them less suitable for real-time deployment, especially in dynamic traffic scenarios. These limitations have driven a shift toward deep learning-based methods, which automatically learn hierarchical feature representations from data and offer greater robustness and generalization [53]. The following section explores these modern techniques in detail, highlighting their advantages and recent developments in vehicle detection.

2.2. Deep Learning-Based Vehicle Detection Methods

Deep learning-based vehicle detection methods have transformed vision-based systems by leveraging convolutional neural networks (CNNs) to automate feature extraction and object recognition. Deep learning, a subfield of machine learning inspired by the hierarchical processing of the human brain, learns abstract representations from raw data through multiple nonlinear layers [54]. Figure 5 illustrates a typical deep learning workflow for object detection.

While deep models require large annotated datasets and significant computational resources for training, they deliver strong generalization performance and fast inference once deployed. Several models have shown outstanding results in vehicle detection tasks [55,56,57,58].

2.2.1. One-Stage Detectors

One-stage detectors such as YOLO [23,24,25,26,27,59], SSD [60], and RetinaNet [61] perform detection in a single feed-forward pass. These models are designed for speed and efficiency, making them ideal for real-time vehicle detection. Their architecture typically includes a backbone for feature extraction, a neck for multi-scale feature fusion, and a detection head for simultaneous classification and localization. Several YOLO-based methods improved anchor box initialization by replacing the traditional k-means clustering with more effective variants. For example, in [62,63], the k-means++ algorithm was used to generate anchor boxes that better match the distribution of vehicle sizes in the training dataset, enhancing localization accuracy and convergence speed. Furthermore, ref. [64] introduced a refined method called Rk-means++, designed to accommodate class imbalance and varying object scales, resulting in improved multi-scale detection performance. These adaptive clustering strategies enable more accurate bounding box proposals tailored to diverse vehicle dimensions in real-world scenarios. Figure 6 depicts a general one-stage pipeline.

• YOLO Series Algorithms
  Object detection in real-time applications requires high-speed, accurate algorithms. YOLO (You Only Look Once) was introduced to meet real-time processing demands, with YOLOv1 [23] pioneering the first one-stage object detection algorithm. This series, including YOLO versions 2 and 3 [24,25], opened new opportunities for vehicle detection. Significant improvements have continued with YOLOv4 [59], YOLOv5 [65], YOLOv6 [26], YOLOv7 [27], YOLOX [66], PP-YOLOE [67], and the latest versions of YOLO including YOLOv8, YOLOv9, and YOLOv10 [68,69,70]. The YOLO architecture typically comprises the following three key components:

  -

  Backbone: Responsible for extracting low-level and high-level visual features from the input image.

  -

  Neck: Connects the backbone to the head and enhances spatial and semantic information across different scales using feature fusion modules such as PANet, BiFPN, or path aggregation blocks.

  -

  Head: Generates output predictions, including object classification, bounding box regression, instance segmentation, or pose estimation. Non-maximum suppression (NMS) is applied post-processing to remove redundant overlapping boxes.

  Over the years, the YOLO series has evolved significantly, from YOLOv1 to the most recent YOLOv12 [65,68,69,70,71,72], with progressive improvements in accuracy, speed, and computational efficiency. These advances have made YOLO models increasingly suitable for real-time applications on edge devices.

  Table 3. YOLO Series Algorithms for Real-Time Object Detection.

  Reference	Year	Version	Backbone	Head
  [23]	2015	YOLOv1	Custom	Custom
  [24]	2016	YOLOv2	Darknet-19	Custom
  [25]	2018	YOLOv3	Darknet-53	Custom
  [59]	2020	YOLOv4	CSPDarknet53	Custom
  [65]	2020	YOLOv5	CSPDarknet	Custom
  [26]	2022	YOLOv6	EfficientRep	Custom
  [27]	2022	YOLOv7	YOLOv7Backbone	Custom
  [68]	2023	YOLOv8	YOLOv8CSPDarknet	Custom
  [69]	2024	YOLOv9	YOLOv9Backbone	DEKRHead
  [70]	2024	YOLOv10	YOLOv10Backbone	YOLOv10Head
  [71]	2024	YOLOv11	C3k2 + C2PSA	YOLOv11Head
  [72]	2025	YOLOv12	R-ELAN + A2C2f	YOLOv12Head

  and Figure 7 summarize the architectural progression across different YOLO versions.
• Single Shot MultiBox Detector (SSD) algorithm [60] employs a feed-forward convolutional network to generate a fixed set of bounding boxes and confidence scores for object classes within those boxes. A non-maximum suppression step is then applied to refine the final detections. The SSD algorithm comprises two main components: the first extracts feature maps using the VGG16 network [73], and the second uses convolutional filters for object detection. As a one-stage object detection algorithm, SSD achieved a mean average precision (mAP) of 76.8 % on Pascal VOC 2007 at a speed of 19 frames per second (FPS), making it well-suited for real-time object detection due to its faster performance compared to two-stage algorithms. However, its accuracy is lower than those of two-stage detectors such as the Faster R-CNN series. Several lightweight backbone networks [74,75,76] have been developed to enhance the SSD algorithm’s Backbone. For instance, Chen et al. [77] optimized the SSD model to improve vehicle detection speed. They introduced an attention mechanism and bottom-up feature fusion using a deconvolution module by replacing VGG16 with MobileNetv2 [76] as the Backbone for feature extraction. Similarly, Zhang et al. [78] proposed an improved SSD-based model that enhances vehicle detection performance, enabling real-time detection of various types of vehicles. Table 3 provides a summary of different one-step approaches for vehicle detection tasks.
  SSD [60], originally built on VGG16, has benefited from lightweight backbones like MobileNet [76]. Enhancements such as attention modules and multi-scale feature layers have further improved detection accuracy [77,78]. Anchor-free methods like FCOS and CenterNet [79,80] remove the need for manually designed anchor boxes, simplifying training, which significantly enhances inference speed—an essential requirement for autonomous driving systems that rely on real-time perception.

While CNN-based one-stage detectors dominate real-time tasks, transformer-based models offer improvements in global context modeling, especially in complex urban scenes.

2.2.2. Transformer-Based Methods

Transformer-based architectures such as DETR [81], Deformable DETR [82], and Swin Transformer [83,84] offer global context modeling and strong performance in dense or cluttered scenes. These models leverage attention mechanisms to improve spatial reasoning, making them well-suited for complex urban traffic environments.

2.2.3. Gan-Based Methods

Generative Adversarial Networks (GANs) enhance vehicle detection under challenging visual conditions such as low light, occlusion, or limited data availability. Models like VAGAN [85], AugGAN [86], and VS-GAN [87] are used to augment datasets, simulate diverse conditions, or restore degraded images, thereby improving the robustness of vehicle detectors.

Table 4. Examples of one-stage methods for vehicle detection.

References	Dataset	Method	Performance
[78]	UA-DETRAC	DP-SSD	77.94%
[77]	KITTI	SSD + (MobileNet v2, channel attention, deconvolution)	84.81%
[88]	MS COCO	YOLOv5s, C3Ghost, CBAM	72.40%
[89]	PASCAL VOC 2007 + 2012	YOLOv4, MobileNetv1, ECA attention	90.29%
[62]	KITTI	YOLOv4, CBM + CSP, Kmeans++	93.22%
[63]	BIT-Vehicle, CompCars	YOLOv2, k-means++	94.78%
[64]	BIT-Vehicle	YOLOv2, Rk-means++, Focal Loss	97.30%
[90]	PASCAL VOC, MS COCO, PKLot	YOLOv3	93.30%
[55]	PASCAL VOC 2007 + 2012	YOLOv3	89.10%
[91]	Vehicle dataset	YOLOv3, ORB algorithm	87.80%
[92]	KITTI, PASCAL VOC	YOLOv4	97.70%
[93]	MAFAT tournament	YOLOv3	81.72%
[94]	DETRAC	YOLOv3 with SPP module	98.80%
[95]	BIT-Vehicle	YOLOv2-tiny	82.20%
[96]	KITTI	Complex-YOLO, E-RPN	67.70%
[97]	KITTI	Complex-YOLO V4	79.00%
[7]	KITTI, BDD	YOLOv4, CSPDarknet53_dcn, PAN++	70.10%
[98]	KITTI	YOLOv2	61.30%
[99]	KITTI	YOLOv4, CBAM, Soft-NMS, DIoU	81.23%
[5]	PoribohonBD, Dhaka-AI	YOLOv4	87.19%
[100]	BIT-Vehicle	YOLOX	99.21%
[101]	VOC2007	YOLOv2	90.00%

summarizes several prominent one-stage vehicle detection methods evaluated across different datasets.

2.2.4. Two-Stage Detectors

Two-stage detectors, including Faster R-CNN and Mask R-CNN, operate in two sequential steps: region proposal and object classification/regression [30,102]. Though more computationally intensive, they are favored for their high accuracy and reliability in scenarios where precision is paramount. Figure 8 depicts the two-stage detection pipeline.

Numerous studies have applied two-stage frameworks for vehicle detection across various datasets.

Table 5. Examples of vehicle detection methods based on two-stage detectors.

Reference	Dataset	Method	Performance
[103]	UA-DETRAC	R-CNN, Transfer Learning	93.43%
[104]	SHRP 2 NDS	Fast R-CNN	89.2%
[102]	UAV Images	Mask R-CNN	93%
[105]	Pakistan Dataset	Faster R-CNN	82.14%
[106]	CIFAR-10	R-CNN	98.5%
[107]	PASCAL VOC	Optimized Faster R-CNN	90%
[108]	KITTI, LSVH	Faster R-CNN, Soft-NMS	89.2%

provides examples and their reported performance, and

Table 6. Comparison of deep learning-based vehicle detection methods.

Method	Model Type	Application Scenario	Strengths
YOLOv5/YOLOv7	One-Stage	Real-time/Edge	Speed, Efficiency
Faster R-CNN	Two-Stage	High-Precision Detection	Accuracy, Localization
DETR/Swin	Transformer	Complex Scenes	Context Modeling
VAGAN/AugGAN	GAN-Based	Low-Light/Scarce Data	Data Augmentation

shows a comparison of deep learning-based vehicle detection methods.

2.2.5. Summary of Deep Learning Vehicle Detection Method

Deep learning-based vehicle detection methods span a diverse set of architectures, each tailored to specific performance requirements and deployment environments. One-stage detectors such as YOLO and SSD offer fast inference and are well-suited for real-time applications, particularly on resource-constrained devices. In contrast, two-stage detectors like Faster R-CNN and Mask R-CNN deliver higher detection accuracy, making them preferable in precision-critical tasks. Transformer-based models enhance global context modeling and spatial reasoning, improving performance in complex or cluttered scenes. GAN-enhanced approaches further boost robustness by augmenting training data or restoring degraded visual inputs. Together, these advances have enabled state-of-the-art vehicle detection systems that generalize well across diverse environments and operational conditions.

3. Application Areas

Vehicle detection serves as a core enabler for numerous modern applications, including traffic control, intelligent transportation systems (ITS), smart city infrastructure, surveillance, and industrial automation. Its significance lies in enabling real-time decision-making, adaptive traffic responses, and enhanced transportation safety and efficiency. In traffic management, vehicle detection supports congestion mitigation through dynamic signal control and optimized route planning. It also enables the timely detection of incidents such as accidents or stalled vehicles, improving emergency response times [88,109]. Detection systems deployed on roadside infrastructure or UAVs offer wide-area monitoring and real-time analytics, enhancing the capabilities of traffic surveillance platforms [110]. In law enforcement and surveillance, vehicle detection is critical for tasks such as automatic number plate recognition (ANPR), speed enforcement, and red-light violation detection. It is increasingly integrated into smart camera networks and tunnel monitoring systems to improve public safety [111,112]. Real-time detection algorithms, including YOLOv4, have demonstrated high accuracy in traffic offense monitoring applications [5]. Within ITS, vehicle detection underpins autonomous driving and advanced driver assistance systems (ADAS). Deep learning-based detectors enable reliable object recognition, allowing autonomous vehicles to perceive their surroundings, avoid collisions, and make safe navigation decisions [113]. These functions rely on real-time detection frameworks that maintain both high accuracy and low latency in dynamic environments. In the context of smart cities, vehicle detection supports intelligent parking management, automated toll collection, and real-time traffic data analytics for urban planning. The resulting data contribute to reduced emissions, improved public transport planning, and more efficient infrastructure use [114,115]. Security applications also benefit from vehicle detection, including restricted zone monitoring, suspicious vehicle tracking, and real-time surveillance in critical infrastructure and sensitive areas [116]. In industrial environments such as warehouses and manufacturing facilities, detection systems improve coordination and safety among autonomous mobile robots and automated guided vehicles (AGVs), ensuring efficient and collision-free operations. Despite its broad applicability, real-world deployment of vehicle detection still presents challenges. One-stage models like YOLO and SSD are commonly used in real-time applications such as UAV-based surveillance and traffic monitoring due to their low latency. In contrast, two-stage models like Faster R-CNN and Mask R-CNN deliver higher accuracy and are preferred in offline processing or safety-critical tasks. Detection performance may degrade under occlusion, low lighting, or adverse weather conditions, highlighting the need for robust and context-aware model selection. Future advancements are expected to focus on integrating multi-modal sensor data (e.g., camera, LiDAR, radar), optimizing models for edge deployment in resource-constrained environments, and adopting transformer-based architectures for improved spatiotemporal reasoning. Furthermore, progress in unsupervised and self-supervised learning could reduce dependence on large labeled datasets, accelerating adoption in emerging smart city and industrial applications.

4. Datasets for Vehicle Detection Models Design

Datasets play a critical role in developing and evaluating object detection models. They provide the training data needed for learning robust features and serve as benchmarks for comparing algorithmic performance [117]. Building high-performing vehicle detection systems requires access to large, diverse, and well-annotated datasets [118]. Several benchmark datasets, such as BDD100K, KITTI, and MS COCO, have been widely adopted to support the advancement of real-time vehicle detection models [62,77].

Table 7. Summary of common datasets for vehicle detection tasks. “#Img/Vid” refers to the approximate number of labeled images or video sequences.

Dataset	Classes	Vehicle Types	Tasks	Annotation	#Img/Vid	Resolution
Pascal VOC	20	Car, Bus, Bicycle, Motorbike	Detection, Segmentation, Classification	Yes	∼11,000 images	384 × 480 to 500 × 375
MS COCO	91	Bicycle, Car, Motorcycle, Bus, Truck	Detection, Segmentation, Classification	Yes	∼330,000 images	Varies (avg. 640 × 480)
BDD100K	10	Car, Bus, Truck, Bike, Motorcycle	Detection, Segmentation, Lane Marking, Image Tagging	Yes	100,000 video clips (40 s each)	1280 × 720
KITTI	9	Car, Van, Truck	Detection, Segmentation, Classification	Yes	∼15,000 images, 200 videos	1242 × 375
Boxy Vehicle	1	Passenger Cars, Trucks, Car Carriers, Motorcycles	Vehicle Detection in Freeway Driving	Yes	200,000 images	5 MP (typically 1920 × 1080)

summarizes key characteristics of commonly used datasets in this domain.

• Pascal VOC [117]: A foundational dataset with 20 object categories, including vehicles like cars, buses, bicycles, and motorcycles. It was used in early versions of YOLO (v1 and v2) and supports tasks such as object detection, segmentation, and classification.
• MS COCO [119]: A large-scale benchmark containing 91 object classes with five dedicated vehicle categories. It is widely used for training state-of-the-art models such as Faster R-CNN and the YOLO series.
• BDD100K [120]: A diverse driving dataset with 100,000 video clips captured under varied conditions. It includes ten classes and supports multiple tasks, including detection, segmentation, and lane marking. It is suitable for complex real-world driving environments.
• KITTI [121]: A widely used dataset for autonomous driving, providing both 2D and 3D annotations for object detection, tracking, and stereo vision. It includes real-world traffic scenes captured from a moving vehicle.
• Boxy Vehicle Detection [122]: One of the largest public datasets for freeway vehicle detection, containing 200,000 annotated images. It covers various weather conditions, traffic densities, and vehicle types, supporting large-scale training and evaluation.

Despite the richness and scale of these datasets, several limitations can hinder the performance and generalizability of vehicle detection models. One common issue is class imbalance, where certain vehicle types (e.g., cars) are overrepresented compared to others (e.g., buses or motorcycles), leading to biased model predictions. Additionally, many datasets lack diversity in environmental conditions, such as nighttime, fog, rain, or occlusion, which are critical for robust real-world deployment. Limited annotation quality, resolution variability, and dataset-specific biases can further affect a model’s ability to generalize to unseen domains or cities. Addressing these challenges requires the development of more balanced, high-resolution, and context-rich datasets, along with techniques such as domain adaptation, data augmentation, and synthetic data generation, to improve model robustness across diverse operational scenarios.

Table 8. Common dataset limitations and mitigation strategies in vehicle detection.

Limitation	Mitigation Strategy
Class imbalance (e.g., overrepresentation of cars)	Data resampling, synthetic data generation, loss reweighting
Poor diversity in weather/lighting	Data augmentation (e.g., rain, fog simulation), use of GANs
Low-quality or inconsistent annotations	Annotation refinement, human-in-the-loop labeling, weak supervision
Limited coverage of rare scenarios (e.g., accidents, occlusion)	Scenario simulation tools, collection of edge-case datasets
Domain-specific bias (e.g., specific cities or camera angles)	Domain adaptation, transfer learning, cross-dataset evaluation

shows some common dataset limitations and corresponding mitigation strategies.

5. Evaluation Metrics

Evaluation metrics are crucial for quantifying the performance of object detection algorithms. A core challenge in object detection is correctly associating predicted bounding boxes with their corresponding ground truth annotations, which significantly affects the calculated performance. Most evaluation protocols rely on Intersection over Union (IoU) to determine the quality of this association. Based on predefined IoU thresholds, standard metrics such as precision, recall, average precision (AP), and mean average precision (mAP) are computed to assess detection effectiveness. Several studies (e.g., [77,88]) have adopted these metrics to validate their vehicle detection frameworks. However, performance evaluation becomes increasingly difficult in complex scenarios involving occlusions, poor lighting, and dense traffic conditions. To address this, recent approaches such as Optimal Transport Assignment (OTA) and SimOTA [66,123] have reformulated label assignment as a global optimization problem, improving robustness and reducing localization errors in such challenging environments. The key metrics commonly used in object detection are described below:

• Precision (P): Measures the proportion of correctly predicted positive detections:

  $$P={\displaystyle \frac{TP}{TP+FP}}$$

  (12)

  where $TP$ is the number of true positives and $FP$ the number of false positives.
• Recall (R): Indicates the proportion of actual positives that are correctly detected:

  $$R={\displaystyle \frac{TP}{TP+FN}}$$

  (13)

  where $FN$ denotes the number of false negatives.
• Intersection over Union (IoU): Represents the ratio of the overlap between predicted and ground truth bounding boxes to their union:

  $$IoU={\displaystyle \frac{|B_p\cap B_{gt}|}{|B_p\cup B_{gt}|}}$$

  (14)

  where $B_p$ is the predicted bounding box and $B_{gt}$ is the ground truth box. A detection is considered correct if IoU exceeds a predefined threshold (e.g., 0.5).
• Average Precision (AP): Quantifies the area under the precision-recall curve for a given class:

  $$AP={\int }_0^{1}p\left(r\right)dr$$

  (15)

  where $p\left(r\right)$ is the precision at recall level r. It captures the trade-off between precision and recall across different confidence scores.
• Mean Average Precision (mAP): The average of AP values over all object classes:

  $$mAP={\displaystyle \frac{1}{N}}\sum _{i=1}^{N}A{P}_i$$

  (16)

  where N is the number of classes and $A{P}_i$ is the AP for class i. mAP is widely used to compare detection models comprehensively.

Model Optimization for Extreme Real-World Conditions

While single-stage detectors such as the YOLO series are renowned for their real-time inference capability, their performance often deteriorates under challenging real-world conditions, including occlusion, low lighting, and dense traffic. To address these issues and maintain high-frame-rate performance with robust accuracy, several optimization strategies have been proposed in recent literature.

• Lightweight Backbone Integration: Efficient architectures such as MobileNet and GhostNet are increasingly adopted to enhance computational efficiency without significantly compromising detection accuracy. Models like MobileNet-SSDv2 deliver fast inference on embedded platforms with low power consumption [124]. Similarly, GhostNet-based variants, including GhostNet-SSD [125] and GS-YoloNet [126], reduce redundant computation by generating ghost feature maps, preserving spatial richness.
• Multi-Feature Fusion and Attention Mechanisms: Architectures such as SYGNet employ semantic-visual-guided GhostNet-YOLO to improve real-time detection in occluded or cluttered driving scenarios by fusing spatial and contextual information [127]. GS-YoloNet further enhances robustness using GhostShuffle and attention-based fusion techniques [126].
• Hybrid Architecture Strategies: Combining YOLOv4 with EfficientDet modules has shown improved detection performance under lighting variation and visual clutter. This is achieved through BiFPN and compound scaling, which balance detection accuracy and computational efficiency [8].
• Hardware-Aware Acceleration: TensorRT-optimized models such as YOLOv8-QSD demonstrate exceptional performance in low-light and small-object scenarios by leveraging quadrant spatial distribution encoding and depth-aware feature fusion [128].
• Backbone Benchmarking for Deployment: Comparative studies highlight that lightweight backbones (e.g., MobileNet) offer favorable trade-offs over heavier ones (e.g., ConvNeXt) in terms of speed, energy efficiency, and accuracy for edge deployments in vehicle detection tasks [129].

6. Challenges and Future Research Directions

Despite considerable advancements in vehicle detection, several persistent challenges hinder reliable deployment in real-world environments. Key challenges and prospective research directions include the following:

• Dataset and Annotation Quality: Accurate and diverse labeling is essential for training effective vehicle detection models. Labeling errors or imbalances—particularly in single-stage detectors—can lead to biased learning and degraded performance [78].
• Complex Traffic Scenarios: Real-world traffic involves frequent occlusions, scale variations, and overlapping vehicles. Recent studies show that attention mechanisms—including spatial, channel-wise, and transformer-based—substantially improve robustness in these scenarios. Transformer models like RT-DETR and ViT variants capture long-range dependencies, while convolutional attention modules enhance feature discrimination in cluttered scenes [12,18,81,82].
• Environmental Variability: Illumination changes, shadows, and adverse weather (e.g., fog, rain) remain major obstacles. Solutions include synthetic data augmentation, domain adaptation, and deep sensor fusion techniques, which improve detection reliability under degraded visual conditions [130,131].
• Resource-Constrained Deployment: Real-time applications, particularly on UAVs or embedded systems, demand high efficiency. Lightweight models (e.g., YOLOv5, YOLOv6) combined with model compression techniques—such as pruning and quantization—are actively explored to address these constraints [88].
• Generalization and Cross-Domain Adaptability: Many models experience performance drops when applied to unseen datasets or environments. Transfer learning, data augmentation, and continual learning are promising techniques to enhance model adaptability [114,115].
• Integration with V2X Communication: The integration of detection systems with Vehicle-to-Everything (V2X) networks can enhance cooperative perception and situational awareness. Future work should explore detection architectures capable of leveraging shared sensor data across connected vehicles and infrastructure [110]. Such multi-agent cooperation frameworks, as discussed in [11], depend critically on the reliability and timeliness of vehicle detection outputs, underscoring the importance of accurate, low-latency perception in MAS-based driving environments.

7. Conclusions

This paper has presented a comprehensive survey of vehicle detection methods developed over the past two decades. We began by reviewing classical approaches rooted in traditional machine learning, highlighting widely used feature extraction techniques such as Haar-like features, HOG, and LBP, in combination with classifiers like SVM and AdaBoost. Representative studies were discussed in terms of detection tasks and performance, illustrating both the capabilities and limitations of early methods. We then examined the evolution toward deep learning-based approaches, including one-stage detectors (e.g., YOLO, SSD, FCOS, CenterNet) and two-stage frameworks (e.g., Faster R-CNN, Mask R-CNN). Emphasis was placed on architectural advancements, dataset usage, and performance trade-offs, especially regarding accuracy and real-time inference. Comparative tables (e.g., Table 1 and Table 5) offer practical insights into model selection and deployment considerations. In addition, we explored emerging directions such as transformer-based and GAN-enhanced detection models, integration with V2X communication systems, and edge–cloud deployment strategies. Major challenges, such as dense traffic scenes, adverse environmental conditions, limited annotations, and resource constraints, were also discussed. Looking ahead, future research should focus on enhancing model generalization, improving robustness via multi-modal and cooperative perception, advancing domain adaptation techniques, and enabling efficient real-time deployment on edge devices. This survey aims to serve as a valuable reference for researchers and practitioners seeking to understand the historical development, current state, and future potential of vehicle detection technologies.