AUTOMATIC TRAFFIC DENSITY ESTIMATION AND VEHICLE CLASSIFICATION FOR TRAFFIC SURVEILLANCE SYSTEMS USING NEURAL NETWORKS

-It is important to know the road traffic density real time especially in mega cities for signal control and effective traffic management. In recent years, video monitoring and surveillance systems have been widely used in traffic management. Hence, traffic density estimation and vehicle classification can be achieved using video monitoring systems. In most vehicle detection methods in the literature, only the detection of vehicles in frames of the given video is emphesized. However, further analysis is needed in order to obtain the useful information for traffic management such as real time traffic density and number of vehicle types passing these roads. This paper presents vehicle classification and traffic density calculation methods using neural networks. The paper also reports results from real traffic videos obtained from Istanbul Traffic Management Company (ISBAK).


INTRODUCTION
In recent years, video monitoring and surveillance systems have been widely used in traffic management.For example, Istanbul Traffic Management Company (ISBAK) have started to use more than 500 cameras for traffic monitoring.Extracting useful information such as traffic density and vehicle types from these camera systems has become a hassle due to the high number of cameras in use.Manual analysis of these camera systems is now unapplicable.Development of intellegent systems that extract traffic density and vehicle classification information from traffic surveillance systems is crucial in traffic management.
It is important to know the traffic density of the roads real time especially in mega cities for signal control and effective traffic management.Time estimation of reaching from one location to another and recommendation of different route alternatives using real time traffic density information are very valuable for mega city residents.In addition, vehicle classification (big: truck, middle: van, or small: car) is also important for traffic control centers.For example, the effects of banning big vehicles from a road can be analyzed using vehicle classification information in a simulation program.This paper presents an automatic traffic density estimation and vehicle classification method for traffic surveillence system using neural networks.
Several other vehicle detectors such as loop, radar, infrared, ultrasonic, and microwave detectors exist in the literature.These sensors are expensive with limited capacity and involve installation, maintenance, and implementation difficulties.For example, loop sensor might need maintenance due to road ground deformation or metal barrier near the road might prevent effective detection using radar sensors [1].In resent years, video processing techniques have attracted researchers for vehicle detection [2][3][4][5][6][7].
Detection of moving objects including vehicle, human, etc. in video can be achieved in three main approaches: Temporal difference, optical flow, and background substraction.In temporal difference, the image difference of two consecutive image frames are obtained [12][13][14][15][16][17][18].However, this approach has some limitations such as visual homogeneity requirement and its effectiveness depends on the speeds of moving objects [2].Optical flow method was developed to obtain effective background modification, which bases on the detection of intensity changes [2].However, illumunation change due to weather or sun-light reflections decreases its effectiveness.It is also computationally inefficient [2].The third method, background subtraction, is the mostly seen method in the literature for effective motion tracking and moving object identification [2,4,6,9,10,11].In background subtraction, background can be static, in which a fixed background is obtained beforehand and used in the entire process; or dynamic, in which background is dynamically updated with changing external effects like weather.Static background may not be effective in most applications, many methods include dynamic background subtraction.In [19], the background is detected dynamically by using dynamic threshold selection method.In [22], land mark based method and BS&Edge method are used to remove the shadow from the scene.Different classification techniques have been employed after the moving objects are detected in order to identify the moving object.In [4], support vector machines is used to idenfity if the detected moving object is a vehicle or not.Support vector machine is a two class classification method and requires modification for multi class classification.The vehicles are detected using mathematical modeling in [21].The expected parameters of a moving vehicle is matematically modeled using the position of the camera, vechile, and sun; it is compared with the values obtained from the video.However, this model requires very sensitive calibration of the camera and it works for cases with short distance between camera and vehicles.The traffic videos used in Istanbul do not satisfy these needs.In [20], rule based reasoning is used for vehicle detection, in which the results highly depend on the rules decided by humans.
Neural networks have been widely used in traffic control [23,24,25].In [23], the traffic incident detection model using neural networks has been developed using traffic magnetic sensors.Intelligent agent systems have been using in [24] in order to control the traffic.Hybrid computational intelligent techniques and fuzzy neural networks has been applied in multi agents in order to control the traffic signals.They have reduced the average waiting time in the traffic.In [25], traffic flow prediction is achieved using time delay based neural networks.In this paper, we will use neural networks for identification of vehicles and traffic density by processing the traffic videos.
In most vehicle detection methods in the literature, only the detection of vehicles in frames of the given video is emphesized.However, further analysis is needed in order to obtain the useful information for traffic management such as real time intensity of roads and number of vehicle types passing these roads.This paper presents vehicle classification and road intensity calculation methods using neural networks.Section 2 discusses the modeling background, section 3 gives the traffic density estimation algorithm and results obtained from traffic videos in Istanbul.Section 4 concludes the paper.

MODELING BACKGROUND
Our model consists of three submodels: Moving Object Detector (MOD), Vehicle Identifier (VI), Traffic Density Calculator (TDC) as shown in Fig. 1.In MOD, moving objects are detected using background estimation method.In VI, the vehicles are detected and identified as small, middle, and big using neural network model.In TDC, the traffic density is calculated using the identified vehicle information in successive frames.The details of these submodels are explained in following subsections.Moving object detection is applied to each frame distinctly and performed as shown in Fig. 3.In the first step, the background of the video is subtracted from the current image and the a treshold is applied to the difference matrix.Then, the moving object is detected by analyzing the difference matrix between background and the current frame image.Lastly, the background is updated with the current frame for the following frames.In the first step, the background is subtracted from the current frame.The difference matrix is applied to a threshold.The gray levels greater and lower than the threshold is updated as 1 and 0, respectively, which leads moving objects to be represented as white pixels as shown in Fig. 3.b and formulated as in ( 1) and (2).Once the binary matrix showing the difference between current frame and background is obtained, this matrix is analyized in order to detect the moving object.The biggest area of ones within this matrix is detected.For example there are three objects represented with ones in the matrix given in Fig. 4. In order to classify the moving objects, as much information as possible should be extracted.For this purpose, smallest ellipse and rectangle that covers the given object is considered and 14 properties are identified for each object.These properties are given in Fig. 5.As mentioned earlier, the static background (e.g., the image of the road without any vehicle is passing on the road) may be used.However, this approach works only for cases where external factors such as weather, illumnuation (day, night or time within a day) effects are the same with the static background.Thus, dynamic background calculation methods are common in the literature.In the dynamic background calculation, the background is updated for each frame.In our algorithm, the pixel gray levels are stored as time series data and the median value of the time series data is updated based upon the new gray level in the current frame.

Vehicle Identificator (VI)
In this work feed-forward Neural Network is used.Neural Networks (NN) have been used for decades for solving problems such as classification, clustering, and function approximation.NNs inspired from biological brains consisting of millions of interconnected neurons.NN calculates an output by processing the inputs through neurons in input, hidden, and output layers.There exist several types of NN methods.
Our Neural Network model consists of 14 input and 4 output layers.Input layers are the 14 object properties identified in MOD and shown in Fig. 5. Output layers are binary valued nodes and each represent a vehicle type (i.e., big, small, and medium vehicle and not a vehicle).Only one output node can be one at a time.In other words, these output nodes compete each other in order to represent the given input.Only one hidden layer is used, which includes 25 nodes.It is optimized using trial-error method.Fig. 6 represents the structure of the neural network model.In the first two submodels (MOD and VI), the frames are processed individually.In this submodel, all video frames for a given time period is processed together in order to calculate the number of vehicles that passed through the road for the given time period.Succusive frames represent the scene of the road miliseconds after each other.Thus, the same vehicles will be seen in successive frames.In this submodel, number of vehicles that passed the road for the given time period is counted using the location of the vehicle in successive frames.The traffic density is calculated as the number of vehicles over the time as shown in (3).The number of vehicles is calculated according to the algorithm given in Fig. 7. Isclose=false; 1 Store the selected vehicle for future comparisons

MODEL APPLICATION TO REAL VIDEOS
The presented method is applied to video obtained from one of the traffic cameras used Istanbul traffic management company.One scene of the video is given in Fig. 8.In the selected video, there are three road parallel to each other.Each road consists of two lanes.The left road, which is the most crowded one, is selected in our application.Our model is applied to 1000 frames, which lasts 100 seconds (10 frames in a second).Since we are interested in only one road, the rest of the matrix is cut out and the background is calculated only for the selected road as in Fig. 9.The difference matrix is used to identify the vehicle nominees.14 properties shown in Fig. 5. is used as input for NN model.NN model identifies the vehicles with classification accuracy of 98.9878.After the vehicles in frames are identified, traffic intensity is calculated.In 100 seconds, 68 vehicles has passed from the road.Number of vehicles counted by our algorithm is also 68.In other words, traffic intensity is calculated without any error.However, the classification of vehicle types is performed with some error.Some of the vehicles have been misclassified.The results are given in Table 1.
The video is uploaded to the our website at http://www.fatih.edu.tr/~fcamci/proje/isbak/test.html for future studies.

CONCLUSION
Automatic traffic density estimation and vehicle classification through video processing and artificial systems are important for traffic management companies especially in mega cities. Traditional traffic density estimation methods such as radars, loop sensors, ultrasonic waves etc. have some limitations.In this paper, automatic traffic density estimation and vehicle classification method using neural network is presented.The presented method is applied to real traffic videos used in Istanbul Traffic Management Company (ISBAK).The results are promising and presented in the paper.

DFigure 3 :
Figure 3: a) Dynamic background b) Difference between the current frame and background

Figure 4 .
Figure 4. Example of Difference Matrix

Figure 5 .
Figure 5. Properties to be used to classify a vehicle.

Figure 8 .
Figure 8.One scene of the traffic video used in model

Figure 9 .
Figure 9.The background image

Table 1 :
Classification Results