Traffic Management: Multi-Scale Vehicle Detection in Varying Weather Conditions Using YOLOv4 and Spatial Pyramid Pooling Network

Abstract

Detecting and counting on road vehicles is a key task in intelligent transport management and surveillance systems. The applicability lies both in urban and highway traffic monitoring and control, particularly in difficult weather and traffic conditions. In the past, the task has been performed through data acquired from sensors and conventional image processing toolbox. However, with the advent of emerging deep learning based smart computer vision systems the task has become computationally efficient and reliable. The data acquired from road mounted surveillance cameras can be used to train models which can detect and track on road vehicles for smart traffic analysis and handling problems such as traffic congestion particularly in harsh weather conditions where there are poor visibility issues because of low illumination and blurring. Different vehicle detection algorithms focusing the same issue deal only with on or two specific conditions. In this research, we address detecting vehicles in a scene in multiple weather scenarios including haze, dust and sandstorms, snowy and rainy weather both in day and nighttime. The proposed architecture uses CSPDarknet53 as baseline architecture modified with spatial pyramid pooling (SPP-NET) layer and reduced Batch Normalization layers. We also augment the DAWN Dataset with different techniques including Hue, Saturation, Exposure, Brightness, Darkness, Blur and Noise. This not only increases the size of the dataset but also make the detection more challenging. The model obtained mean average precision of 81% during training and detected smallest vehicle present in the image

1. Introduction

Over the last decade, deep neural networks have greatly influenced our way of working and processing information. Particularly, their ability to perceive, understand and analyze the visual information accurately and quickly has made them an important part of computer vision applications such as image classification [1,2,3,4,5,6,7], semantic segmentation [8,9,10,11,12,13], object detection [14,15,16,17,18,19] and object segmentation [20,21,22,23,24,25,26].

The task of object detection aims to detect instances of different objects for example humans [27,28,29,30], animals [31,32,33], trees [34,35,36,37], vehicles [38,39,40,41] or other items present in digital media such as images and video frames. To automate this task machine and deep learning involves developing computational models which are trained by learning class-specific features using huge amount of annotated data and then localizing and generating bounding boxes around each instance and label them. This makes object detection a super task of many other smaller tasks such as image captioning [42,43,44,45], object tracking [46,47,48,49], instance segmentation [50,51,52] and instance counting [53,54,55,56].

2. Approaches to Object Detection

Object detection methods can be mainly classified into classical methods which relies on manual feature extraction algorithms with a machine learning classifier and the recent more robust methods which depends on deep learning models for automatic feature extraction.

2.1. Object Detection Using Machine Learning

Classical methods of detecting objects within a media involves feature extraction algorithms such as HOG [57,58,59,60,61,62,63], HAAR [64,65,66,67,68,69,70,71,72,73,74] and SIFT [75,76,77,78,79,80,81] to manually craft features. Secondly, using a classifier such as support vector machine [8,82,83,84,85,86] and K Nearest Neighbor [84,87,88] to classify multiple objects within an image. The general processing pipeline is shown in Figure 1.

2.2. Object Detection Using Deep Learning

Deep Learning based algorithms provide a more robust and accurate solution to the problem of object detection. There is no need for a separate feature description part, rather the model is trained using images with bounding boxes and class labels. Thus the model learns automatically the visual attributes present in the underlying image. In the past few years, the task of object detection using deep learning is performed by a number of algorithms including RCNN, Fast RCNN, Faster RCNN, YOLO, YOLOv2, YOLOv3, YOLOv4 and RetinaNet.

2.2.1. RCNN Family

The algorithms belonging to this family are called “Classification Based Algorithms”. The first variant based on AlexNet as CNN backbone, known as Region based Convolution Neural Network or R-CNN was proposed by [89] in 2015. The network works by first identifying the regions with the probability of having objects. The rectangular boundaries are drawn using a selective search algorithm containing many ROI (Region of Interest). All these candidate b ounding boxes are passed into the layers of convolution neural network. The CNN extract individual features from each bounding box. The output vector from the CNN representing the feature map is passed on to the classifier which then classifies each one of them into object and not object. The Fast RCNN [90] is inspired by Spatial Pyramid Pooling Network [91], The limitation of multi staged pipeline architecture which was the reason for slow training was overcome by the same author by applying CNN over whole image only once. The architecture of RCNN and Fast RCNN fron the papers are shown in Figure 2a,b.

The more refined and improved algorithms proposed under this family includes [92,93,94,95].

2.2.2. YOLO Family

YOLO is an abbreviation of “You Only Look Once” which is a regression based algorithm. It is a class of Convolution Neural Network which is meant for fast and accurate object detection in a single go. Soon after it was released in 2015 by [96], it gained world wide attention and many papers have been published on the topic from 2015 till now including [93,97,98,99,100,101]. The basic architecture of YOLO contains DarkNet as a backbone. DarkNet is an open source framework to implement neural networks written in C language and CUDA. It supports CPU and GPU computing, is quick, and is simple to install. The YOLO comprises of CNN layers, where vanilla YOLO by [96] contains 24 conv layers and 2 fully connected layers, whereas Fast Yolo [102] contains 9 convolutional layers.

The algorithm works by dividing an input image into grids of size $SXS$. Each grid is responsible for predicting a number of bounding boxes for an object. Each bounding box is represented by five parameters as represented in Equation (1). Out of many bounding boxes the one with highest confidence score is selected. This is conducted by applying Non Maximum suppression (NMS) over all bounding boxes.

$$Bounding\_Box=\left\{P_{confidence},x,y,w,h\right\}$$

(1)

Here, x and y represents the centre coordinates of the bounding box, w is the Width of the Bounding Box, h is the Height of the Bounding Box and, $P\_confidence$ represents the Probability in terms of Score from 0 to 1 that the Bounding Box contains an object. In addition, all YOLO models share the same architecture, which is broken down into the following components.

• BackBone: It consists of a convolutional neural network that gathers and generates visual features of various sizes and shapes. The feature extractors employed are classification models such as ResNet, VGG, and EfficientNet.
• Neck: A group of layers that combine and mix traits before sending them to the prediction layer.
• Head: Combines the predictions from the bounding box with features from the neck. The features and bounding box coordinates are subjected to classification and regression in order to complete the detection procedure. normally produces 4 values, including width, height, and x, y coordinates.

The other versions of YOLO include;

• YOLOv2: [96] overcomes the limitations in the previous version by replacing the backbone architecture with DarkNet19 which results in faster detection with mAP of 78.6% on 544 × 544 resolution images from PASCAL VOC 2007 dataset. Moreover, out of the some improvements in this version compared to the predecessor, the first one is the addition of batch normalization layer for enhanced stability in the network. Secondly, the replacement of grid boxes with anchor box which allows the prediction of multiple objects. The ratio of overlap over union IoU between the predetermined anchor box and the anticipated bounding box is calculated. The IoU value is a threshold used to determine whether or not the likelihood of the identified object is high enough to make a forecast.

  $$IOU=\frac{area\_of\_overlap}{area\_of\_union}$$

  (2)
• YOLOv3: The same authors published their updated algorithm [103] in 2018. The Darknet-53 architecture served as its foundation. Newer version introduced the idea of Feature Pyramid Network (FPN) for multiple feature extraction for an image.
• YOLOv4: The bag of freebies and bag of specials concepts were introduced in YOLOv4 [104] as methods for improving model performance without raising the cost of inference. However, the author of YOLOv4 did experiment on three different backbone architectures including, CSPDarknet533, CSPResNext50 and EfficientNet-B3. The bag of specials allows the selection of additional modules in the neck such as Special Attention Module (SAM), Special Pyramid Pooling (SPP) and Path Aggregation Network (PAN).

2.2.3. RetinaNet

Introduced by [105], this is one of the best one-stage object detection models and has demonstrated success with dense and small-scale objects.

3. Literature Review

The task of vehicle detection and tracking systems is addressed by many researchers. The research can be classified based on three different criteria as illustrated in Figure 3.

• Classification based on type of input:

  (a)

  surveillance cameras acquired images and videos;

  (b)

  UAV acquired images and videos;

  (c)

  sensors acquired data (e.g., Lidar and Radar).

• Classification based on object detection techniques:

  (a)

  detection using Image processing with machine learning;

  (b)

  deep learning-based systems.

  i

  Two Stage Detector;

  ii

  One Stage Detector;

  iii

  Hybrid Models.

• Classification based on purpose

  (a)

  Autonomous Vehicles and Self driving vehicles

  (b)

  Traffic Surveillance

The process of recognising moving cars on the road can be conducted by using motion based or appearance based methods and is used for vehicle monitoring [106], counts, the average speed of each vehicle, movement analysis, and vehicle classification purposes. One of the potential application is self driving cars and autonomous vehicles [107]. Because of our page limit we restrict our focus only to research relating to images acquired from road surveillance or UAV mounted cameras. In addition, we will more focus on deep learning-based techniques conducted for the purpose of traffic surveillance monitoring and control.

We start our discussion with simple machine learning based vehicle detection systems. Ref. [108] examined three different classifiers for vehicle recognition including KNN, Neural Network, and Decision Tree. After extracting features using Canny Edge Detector.

Ref. [109] presented a cascade of boosted classifiers based on the characteristics of the vehicle images for detecting vehicles in on-road scene images. In their work, the classifier for vehicle detection was constructed utilising Haar-like features and an AdaBoost technique. The closest neighbourhood of the Euclidean distance was then used for the final classification.

Ref. [110] proposed a new vehicle identification model based on YOLOv2 with enhanced speed, accuracy, and predictive ability. To make the predictions over vehicles of varying sizes in the same image the loss function is normalized. In addition, to reduce the model’s complexity the repeated convolutional layers in the final part of the model were removed as they normally deal with multi class in same image, whereas, the vehicle detection task only involves single class.

The idea of hybridizing manual feature extraction with deep learning framework is presented in a work by [111]. In this work, foreground feature detector is used along with morphological segmentation operations as pre-processing step. Kalman filter is used to hand craft position vector. In the last stage R-CNN is used to label the vehicles in the images. In an another research conducted b [112] a three layer architecture is used to detect vehicles in video frames. The first layer draws feature maps from the input video frames. After that, a Region Proposed Network (RPN) is placed which slides over the feature maps and makes bounding boxes containing vehicles. At the final stage a R-CNN based detection network is used to assign class label to each ROI (Region of Interest) [113] supplies input images to an auto encoder layer before passing it to the deep learning framework. The AE neural network rather works as a pre-processing step and extracts more fine-grained robust features. In addition, they add different noise and vary illumination and contrast levels to make it suitable for training under varying weather conditions. After AE, fast RCNN and YOLO v3 was used for training and results comparison. Some other works where filters are applied as preprocessing the images before they are passed to deep learning model include dehazing, masking and thresholding [114], (automatic white balance combined with Laplacian pyramids) AWBLP [115,116]. Motivated by the idea of saving processing through transfer learning [117] applied frozen weights of SqueezNet [118], ResNet50 [93] and EfficientNet [14] to train on DAWN dataset to detect vehicles under six different weather conditions. Ref. [119] uses YOLOv3 pretrained model on custom acquired images taken at nighttime. To enhance the visibility of the images, Multi Scale Retinex algorithm was applied before the convolution operation. This increases the contrast and removes the uneven brightness values in the images. The method obtained 93.66% average precision value which is higher than the single shot detector and Faster RCNN on the same image dataset. Ref. [120] presented a method to detect on road vehicle by modifying the faster RCNN architecture with multiple Region Proposed Networks (RPN). The reason for this is to capture the features of varying sizes vehicles from smaller to very larger, by setting each RPN with different aspect ratio and scale. The average precision of the proposed method with four RPNs is 89.48% on DAWN and 98.20% on CDNet 2014.

None of the above studies attempt to fully solve the issue of detecting different scales of vehicles appearing in a single image under a variety of weather conditions. However, to some extent the task of multi-scale vehicle detection was addressed by [121] using YOLOv3 as backbone architecture. The work suggest an encoding and decoding-based feature pyramid module. Through a straightforward U-shaped structure, this module may produce a high-order multi-scale feature MAP.

Keeping in mind all the above discussed studies, we propose a smart vehicle detection system using YOLOv4 to detect vehicles of varying sizes in images under varying weather conditions. The main contributions of our work in the domain of smart and intelligent vehicle detection are as follows:

• We address the problem of vehicle detection in varying and harsh weather condition where the problem of low visibility is addressed by using a image restoration module.
• We use YOLOv4 as backbone architecture for the detection of vehicles in six difficult weather conditions including haze, sandstorm, snowstorm, fog, dust, rain both in day and night.
• We address the problem of detection far away and small size vehicles through the addition of double Spatial Pyramid Pooling Network after the last convolution layer [122].
• For capturing vehicle’s fine grained information we add an extra attention module before the convolution layer in the backbone architecture.

4. Materials and Methods

In this section, we discuss in detail the main key processes involved in our research. Figure 4 shows the overall flow of our proposed methodology.

4.1. Dataset

For the research, we chose to train our model with DAWN dataset which contains a collection of 1000 photos from actual traffic scenarios including highways and urban road traffic in varying weather conditions.

4.2. Data Annotation

For the task of object detection, the labeling of images was conducted in YOLO_Darknet format using a free and open source tool ‘LabelImg’ [123]. The labeling was conducted using polygon bounding box. After labeling a text file is placed in the same folder where images are located. The text file contains the {$class\_label,x,y,w,h$} for each image. Here, x and y represents the coordinates of the bounding box (anchor point) and w and h represents the width and height of the bounding box. In our research, we are only detecting cars, so class label is ‘car’. A set of image with assigned bounding boxes and associated text file is shown in Figure 5a,b.

4.3. Data Augmentation

One of the challenging issue in vehicle detection research is the scarcity of available road side data, particularly if we are interested in detecting them in harsh and severe weather situations. Although, the DAWN dataset contains 1000 images from actual traffic scenarios under four weather categories, i.e., fog, sand, rain and snow. However, as the neural network’s model receives increasingly more input, it’s performance can continue to advance. One strategy to overcome this challenge is data augmentation, that allows researchers to greatly broaden the variety of environmental conditions on the road. The dataset was augmented with following techniques.

4.3.1. Hue

This augmentation techniques changes the colour levels in an image randomly so as the model will take into account different colour schemes for objects and scenes in input images. For our dataset images the hue level was chosen between −8° to +82°.

4.3.2. Saturation

Similar to hue augmentation, saturation augmentation modifies how vivid and vibrant the image looks. we augmented the dataset with 50% saturation.

4.3.3. Brightness

To help your model be more resistant to variations in lighting and camera settings, we added variability to the image brightness up to 75%.

4.3.4. Exposure

While generally speaking, the brightness and exposure options both act the same. However, exposure favours highlight tones more than brightness, which has no bias and influences all tones equally. We add exposure in the range −30 to +30.

4.3.5. Blur

To make our model more camera focus-resistant, we added random Gaussian blur upto 10 px. With this the image appears to be completely out of focus due.

4.3.6. Noise

All these augmentation techniques were added to extend the number of training examples set under conditions so as to give an impression of difficult and varying weather conditions such as noise gives the effect of heavy rain and blur gives an impression of haze or fog in the environment. All these augmentation techniques along with original image are shown in Figure 6a–h.

The augmentation techniques are related with the corresponding weather conditions in

Table 1. Relation between augmentation techniques and weather conditions.

	Hue	Saturation	Brightness	Exposure	Blur	Noise
Foggy		- low saturation to create artificial fog			- high blur to show extreeme foggy day
Sunny			- high brightness to show more day time light	- high gamma exposure to create clear visibility
Windy					- adjust gaussian blur to show wind storm
Rainy					- adding Blur to show low visibility during rain	- adding line noise to show falling rain effect
Snowy			- very high brightness to create synthetic snow on road			-adding dot noise to show snow flakes or rain drops
Dusty	- adjust color changes to show dust storm	- low saturation to show poor visibility in dust storm by changing color more vivid
Shady/ Dark			- low brightness to show shade, e.g., underpass, another vehicle shadow	- low gamma exposure to show less visibility at night
Cloudy			-low brightness to show cloudy day
color change	- sunrise - sunset - road illumination and reflection		- lowest values to show night time

.

4.4. Customized Yolov4 Detector

In the section below, we outline our approach which uses YOLOv4 as our baseline architecture. The reason being, high processing frame rate and state-of-the-art accuracy offered by the model as depicted in Figure 7. For the MS COCO, it obtains an accuracy of 43.5% AP (65.7% AP50) with an approximate inference speed of 65 FPS on the Tesla V100 [104].

4.4.1. Overall Model Design

There are a number of parts that make up tour model architecture, but in general they are as follows: The network will be fed a set of training images, which makes up the input, which comes first. These photos are processed by the GPU in concurrent batches. Following the Neck and Backbone, which carry out feature extraction and aggregation, respectively. Our model contains eight convolution layers with 16, 32, 64, 128, 256 and three 512 filters. The object detector is the collective name for the detection head and detection neck. Finally, the head is responsible for detection and prediction. The primary responsibility for detection rests with the head (both localization and classification).

4.4.2. CSPDarknet53 as Backbone Network

To detect multi scale vehicle requires a focused and increased recognition accuracy of a detector. Therefore, we selected CSPDarknet53as as backbone network. The YOLO series deep learning models’ most notable benefit is their ability to swiftly and accurately recognise and classify targets. Figure 8 shows the complete structure of YOLOv4 CSPDarknet.

Figure 9 shows our modified architecture of the network with layered approach.

4.4.3. Adding Spatial Pyramid Pooling Block

From the Bag of specials we add double special pyramid pooling blocks. A pooling layer called spatial pyramid pooling (SPP) frees the network’s fixed-size constraint, meaning a CNN no longer needs a fixed-size input image. In spatial pyramid pooling fine grained features are pooled from any arbitrary sections (sub-images) to create fixed-length representations for training the detectors by computing the feature maps from the complete image just once. By doing this the spatial information in an image is preserved. Hence, the SPP module may significantly expand the receptive field, enhance local and global vehicle recognition accuracy in a complicated environment, and hence enhance detection effectiveness. Figure 10 illustrates the placement of SPP net layer in the overall network.

4.4.4. Removing Batch Normalization

The camera is frequently set up in a specific spot for traffic surveillance purposes. As a result, since the background of photographs is fixed, little preliminary processing and normalisation is required. Hence, by removing the batch normalization layers we are able to reduce model complexity by minimizing the number of required model parameters.

5. Experimental Analysis

5.1. Training Setup

The training of the vehicle detector model was conducted on Google Colab using TensorFlow framework. The reason for this adoption was free availability of GPU back-end. Additionally, adopting a Python code here is simple as most of the required packages are built in.

5.2. Evaluation of the Detector

To evaluate the accuracy of our object detector we took mAP (mean Average Precision) as our main measurement metric. To fully understand mAP we must first comprehend certain fundamental terms such as confidence score, IoU (Intersection over union), precision, recall with respect to our study.

• Confidence Score: It displays the likelihood that bounding box holds a vehicle. It is predicted by the last layer of the detector, i.e., a classifier.
• Intersection over Union (IoU): This metric counts the area encompassed by Actual bounding box (Ba) and predicted bounding box (Bp), We used a predefined threshold value of 0.5 for single class to classify if the prediction is false or true positive.

  $$IoU=\frac{area\_of\_intersection}{area\_of\_union}$$

  (3)

  $$=\frac{area\_of\_B_a\cap B_p}{area\_of\_B_a\cup B_p}$$

  (4)
• Precision: is the ratio of total true positive and sum of true positive and false positive.

  $$precision=\frac{T_p}{T_p+F_p}$$

  (5)
• Recall: is the ratio of true positives and sum of true positive and false negative.

  $$recall=\frac{T_p}{T_p+F_n}$$

  (6)
• Minimum Average Precision ([email protected]):

5.2.1. Results

Table 2. Number of images and car instances in our Augmented DAWN Dataset.

	Training Set	Validation Set	Testing Set
Number of images	1500	424	213
Percentage	70	20	10
Number of car instances	2048	235	101

shows the split of images into training, validation and testing set. The split was conducted using the ratio 70% for training, 15% for validation set and 15% for testing.

One of the main objective of this research was to detect not only clear, large and visible vehicles but also to detect unclear, small size or occluded vehicles. The images in data set contains vehicles of variable sizes. Figure 11a represents the distribution of vehicles with respect to size in the complete dataset using a pie chart. In addition, Figure 11b shows the histogram of vehicle instances per image. As seen in the bar graph, most of the images contains two to three vehicles per image. Around 2000 images contains single vehicle. There were very less images which contains more than six or seven vehicles.

Figure 12 displays the training loss and mean average precision in 2000 epochs with our augmented DAWN Dataset.

Figure 13 shows the comparison of training and validation loss. During experiment it was observed that the training loss of un-augmented DAWN data set was quite high. However, the loss curve was decreased by the data set size through augmentation and fine tuning the hyper = parameter values.

Table 3. Model Hyper parameters set during training.

Name	Value
batch	64
subdivisions	24
momentum = 0.949	0.949
decay = 0.0005	0.005
learning_rate = 0.001	0.001
max_batches = 2000	2000
steps	1600.0, 1800.0

shows the hyper parameter values of our model used during the training.

Figure 14 shows the precision and recall curve of training.

IoU is calculated for each prediction with regard to each ground truth box in the image.

Table 4. Average IoU values calculated by taking 10 random images from each weather condition.

	Fog	Snow	Rain	Sand
Average IoU	0.892	0.914	0.966	0.972

shows the average intersection over union values calculated after taking 10 random sample images from each four categories, i.e., snow, fog, dust and rain and then taking their mean. The results show that almost for each weather condition the average IoU is much greater than >0.5 which is considered as excellent prediction.

5.2.2. Result Comparison and Discussion

The total training time with 2000 images using YOLOv2 detector was 6 h. The Tiny Yolo v3 on the same data set took around 1 and a half hour in training which was comparatively faster than any other detector. The training time with our customized Yolov4 detector was 2.5 h, which was half less than the baseline Yolov4 detector without Spatial pyramid pooling layer.

Table 5. Comparison of mAP of our customized Yolo v4 with three other variants of same family including Yolov2, Tiny Yolov3, baseline Yolov4 and customized Yolo v4.

	Yolo v2 Hour	Tiny Yolov3	Yolov4	Customized Yolo v4
Training Time hours	6	1.5	4	2.5
[email protected] (%)	79	64	89	82

shows the comparison of the training time in hours. and also the mAp (mean average precision) of our model compared with three other variants belonging to same family.

It is evident from the table that our model has optimized training time with good mean average precision of the detector. The Table 5 shows the progression of average precision of our model compared with four other baseline variants. The Figure 15 shows the associated line chart. It is cear that our model reaches the average precision of 82% on the same DAWN Dataset, the YOLOv2, YOLOv3, Tiny YOLOv3 and basic YOLOv4 achieves 57%, 74%, 66% and 80% respectively.

Table 6. Detection of vehicles under different weather conditions with their average precision values.

	Small Vehicles	Medium Size Vehicles	Large Size Vehicles
Fog	78.43	76.45	78.32
Sand	76.35	77.62	79.81
Rain	75.88	76.61	77.34
Snow	77.51	77.23	78.32

shows the average precision score of vehicle detection under the four weather conditions. It can be inferred from the values that large size vehicles have a high precision score, whereas the smallest vehicles have comparatively low precision values still, although these scores are still good.

When we see the testing results as shown in Figure 15a–f, we observe that the prediction scores are quite good and high. Figure 15a shows the far away vehicles in rainy condition have confidence score of 74–81% and a nearby vehicle but occluded with fog have score of 51%. The same fog and low illumination testing accuracy can be observed in Figure 15b, where the bounding box has accuracy of 81%. Figure 15c shows our augmented image with high added noise to make visibility very challenging. However, it can be observed that almost all the vehicles present in the image have been detected with prediction score above 50%. Figure 15d–f shows nighttime images with low illumination, fog and glare respectively. All the vehicles in the images were detected with 60% to 70% prediction score. The last two test images show detection in snow-storm and dust-storm conditions with prediction score of almost 70 to 74%.

6. Conclusions

Based on Yolov4 CSPDarknet53 with SPP-NET, we propose a modified vehicle detector architecture which can detect vehicles in extremely challenging situations. We augmented DAWN Dataset with added noise, blur and varying hue, saturation, brightness and darkness to make the situation more worse. However, it is evident that our detector is able to predict the vehicles which are almost non detectable with the other baseline detectors. Our model achieve the mAP of 81%, which is higher than four other variants of YOLO family including YOLOv2, Yolov3, Tiny Yolo and Yolov4. In addition, the findings indicate that practically every meteorological condition has an average IoU that is significantly greater than >0.5, which is regarded as an excellent forecast. Lastly, we optimized the training time by removing the batch normalization layers since our model was using static background and we added a spatial pyramid pooling layer between the eigth and ninth convolution layer before the first YOLO detector layer. In future, we plan to use generative neural networks for data augmentation and create more realistic synthetic scenes representing challenging on road scenarios and then public the data set. Moreover, we decide to extend the vehicle detection algorithm for vehicle trajectory analysis where using vehicle location, speed, acceleration and jerk, as time functions, normal and abnormal vehicle motion patterns can be detected using video dataset.