Real-Time Traffic Risk Detection Model Using Smart Mobile Device

Automatically recognizing dangerous situations for a vehicle and quickly sharing this information with nearby vehicles is the most essential technology for road safety. In this paper, we propose a real-time deceleration pattern-based traffic risk detection system using smart mobile devices. Our system detects a dangerous situation through machine learning on the deceleration patterns of a driver by considering the vehicle’s headway distance. In order to estimate the vehicle’s headway distance, we introduce a practical vehicle detection method that exploits the shadows on the road and the taillights of the vehicle. For deceleration pattern analysis, the proposed system leverages three machine learning models: neural network, random forest, and clustering. Based on these learning models, we propose two types of decision models to make the final decisions on dangerous situations, and suggest three types of improvements to continuously enhance the traffic risk detection model. Finally, we analyze the accuracy of the proposed model based on actual driving data collected by driving on Seoul city roadways and the Gyeongbu expressway. We also propose an optimal solution for traffic risk detection by analyzing the performance between the proposed decision models and the improvement techniques.


Introduction
Advanced driver assistance systems for driver safety and convenience are growing rapidly and are widely used in the vehicular domain [1]. In particular, various smart phone-based driving assistance systems, including the detection of front vehicles and obstacles, the headway distance, and time to a collision estimation [2][3][4]; the recognition of various driver behaviors and road conditions [5][6][7][8]; the identification of driving styles [9][10][11][12], the detection of emergent braking [13], and an accident alert for preventing a secondary accident [14,15] have been actively proposed. The smart phone-based driving assistant systems are still attractive, since they can be directly used for ordinary vehicles without additional sensor equipment for monitoring driving situations. In this paper, we introduce a smart phone-based driver assistance system that can detect a dangerous situation by machine learning on emergent braking.
Emergent braking or stopping on the road is very dangerous, because it can cause a collision with the following cars. Without alerts on the emergent situation to nearby vehicles, a secondary collision can occur. Thus, a real-time detection of dangerous situations that can be shared with nearby vehicles is a very essential technique for road safety. Our study focuses on the dangerous situation

System Configuration and Operation
Our system consists of a traffic data management server, vehicles, a smart mobile device equipped with a camera and GPS, and our proposed traffic risk detection application. The proposed traffic risk detection application consists of a user interface module, a driving data management module, a risk detection module, and a communication module with the traffic server. The application basically monitors driving behavior by collecting current GPS position and the distance to the front vehicle every second. After estimating the current speed and travel distance, it extracts deceleration segments in real time. Based on the deceleration segments, the learning module performs machine learning regarding normal deceleration and risk deceleration. The decision module detects risk deceleration based on the learning module for new deceleration segments, and generates a warning message when a deceleration segment is considered dangerous. In such a case, the communication module sends the warning message to the traffic server. The traffic server will then feed alarm signals back to the vehicles surrounding the source vehicle in order to prevent a potential secondary accident caused by the following vehicles. Here, the way in which the traffic server selects only vehicles near the source vehicle is beyond the scope of this paper, so we will leave it for our future work. The system illustration of our proposed model is shown in Figure 1. Since it is difficult to define a dangerous deceleration pattern in one sentence, and because such a pattern can occur in various forms depending on the driving situation, we experimentally collected driving data that the driver determined to be dangerous, and used it for machine learning to develop the dangerous pattern in our learning module. Therefore, the neural network, which is a representative labeled-learning method, and the random forest technique, which is suitable for various kinds of classification, are used in parallel. Moreover, since it is necessary to be able to recognize untrained dangerous situations, a clustering technique, which is a representative non-guidance learning model, is used together with these as well. As a result, three learning models are used for our risk deceleration learning.
After establishing an initial risk detection model with the experimentally obtained training data, the decision module determines one of "danger", "suspicious", and "normal" for new deceleration data provided in real time.
Finally, as the driver's real-time driving data is continuously updated, the risk situation detection model must also be continuously updated. Some deceleration segments that are clearly detected as "dangerous" or "normal" are used to train our detection model periodically. Our detection model can thereby be continuously updated and enhanced to a driver-customized model.

Driving Data Management
We will describe the data management module and the risk detection module of the proposed system in further detail in this section. Our proposed vehicle detection and the headway distance estimation will be explained first, and the driving data collection and deceleration segment extraction will then be described in detail.

The Front Vehicle Detection and the Inter-Vehicle Distance Estimation
We basically assume that the smart phone is fixed on the dashboard and the forward image of the vehicle is given as an input. The size of the input image is 960 × 540 in consideration of the screen aspect ratio (16:9) of a general smart mobile device and image processing performance. Our mobile application records 60 frames per second and carries out the vehicle detection and the headway distance estimation every two frames. It is assumed that the taillights of the vehicle are in the red color, and are always located at the rear left and right ends of the vehicle.
Due to the unique characteristics of taillights-that the taillights are commonly in red in all vehicles, and are symmetrically located at both ends of the vehicle-our scheme detects the front vehicle and measures the width of the front vehicle by detecting the taillights of the front vehicle. For an efficient taillight search, an ROI (region of interest) designation is performed first. Since the ROI must include the front vehicle, the ROI is determined based on the shadow of the vehicle. The shadow of the vehicle always appears on the road surface, it is contacted with the wheels of the car, the color is darker than the road surface, and it has a width greater than the width of the car. So, the minimum Since it is difficult to define a dangerous deceleration pattern in one sentence, and because such a pattern can occur in various forms depending on the driving situation, we experimentally collected driving data that the driver determined to be dangerous, and used it for machine learning to develop the dangerous pattern in our learning module. Therefore, the neural network, which is a representative labeled-learning method, and the random forest technique, which is suitable for various kinds of classification, are used in parallel. Moreover, since it is necessary to be able to recognize untrained dangerous situations, a clustering technique, which is a representative non-guidance learning model, is used together with these as well. As a result, three learning models are used for our risk deceleration learning.
After establishing an initial risk detection model with the experimentally obtained training data, the decision module determines one of "danger", "suspicious", and "normal" for new deceleration data provided in real time.
Finally, as the driver's real-time driving data is continuously updated, the risk situation detection model must also be continuously updated. Some deceleration segments that are clearly detected as "dangerous" or "normal" are used to train our detection model periodically. Our detection model can thereby be continuously updated and enhanced to a driver-customized model.

Driving Data Management
We will describe the data management module and the risk detection module of the proposed system in further detail in this section. Our proposed vehicle detection and the headway distance estimation will be explained first, and the driving data collection and deceleration segment extraction will then be described in detail.

The Front Vehicle Detection and the Inter-Vehicle Distance Estimation
We basically assume that the smart phone is fixed on the dashboard and the forward image of the vehicle is given as an input. The size of the input image is 960 × 540 in consideration of the screen aspect ratio (16:9) of a general smart mobile device and image processing performance. Our mobile application records 60 frames per second and carries out the vehicle detection and the headway distance estimation every two frames. It is assumed that the taillights of the vehicle are in the red color, and are always located at the rear left and right ends of the vehicle.
Due to the unique characteristics of taillights-that the taillights are commonly in red in all vehicles, and are symmetrically located at both ends of the vehicle-our scheme detects the front vehicle and measures the width of the front vehicle by detecting the taillights of the front vehicle. For an efficient taillight search, an ROI (region of interest) designation is performed first. Since the ROI must include the front vehicle, the ROI is determined based on the shadow of the vehicle. The shadow of the vehicle always appears on the road surface, it is contacted with the wheels of the car, the color is darker than the road surface, and it has a width greater than the width of the car. So, the minimum area of interest, which can contain the whole vehicle based on the shadow, can be determined. Once the ROI is determined, the taillights of the vehicle are extracted in the ROI, and the width of vehicle can be measured by the distance between the taillights. Finally, the headway distance can be measured by the estimated vehicle width and the angle of view of the camera. The proposed scheme consists of five steps: (1) road area extraction by inverse perspective mapping (IPM) [17], (2) car shadow detection, (3) ROI extraction, (4) taillights detection, and (5) inter-vehicle distance estimation. We will describe the detailed algorithms for each stage in next subsections.
A. Road Area Extraction by IPM (Inverse Perspective Mapping) First, the reduced road area is extracted by applying IPM as a predecessor work for extracting shadows from the road surface. IPM is a method of generating the result of removing the perspective effect by moving the point of the camera to the position vertically above the road, as shown in Figure 2. area of interest, which can contain the whole vehicle based on the shadow, can be determined. Once the ROI is determined, the taillights of the vehicle are extracted in the ROI, and the width of vehicle can be measured by the distance between the taillights. Finally, the headway distance can be measured by the estimated vehicle width and the angle of view of the camera. The proposed scheme consists of five steps: (1) road area extraction by inverse perspective mapping (IPM) [17], (2) car shadow detection, (3) ROI extraction, (4) taillights detection, and (5) inter-vehicle distance estimation. We will describe the detailed algorithms for each stage in next subsections.
A. Road Area Extraction by IPM (Inverse Perspective Mapping) First, the reduced road area is extracted by applying IPM as a predecessor work for extracting shadows from the road surface. IPM is a method of generating the result of removing the perspective effect by moving the point of the camera to the position vertically above the road, as shown in Figure 2. The image provided by the mobile application shows the road surface at the bottom of the center of the image, as shown in Figure 3-(1), except that the distance between the vehicle and the front vehicle is very close. By applying IPM to the road area, the image as shown in Figure 3-(2), which is similar to being captured from the position vertically top from the road, and in which the background area of the image has been removed, can be obtained. The length of the shadow region is relatively different depending on the distance between the vehicle and the front vehicle. This reduces the relative distance difference of the shadow region, so the length of the shadow has a similar length.
(1) The source image (2) IPM applied output image The Algorithm 1 for applying IPM is given below: The image provided by the mobile application shows the road surface at the bottom of the center of the image, as shown in Figure 3-(1), except that the distance between the vehicle and the front vehicle is very close. By applying IPM to the road area, the image as shown in Figure 3-(2), which is similar to being captured from the position vertically top from the road, and in which the background area of the image has been removed, can be obtained. The length of the shadow region is relatively different depending on the distance between the vehicle and the front vehicle. This reduces the relative distance difference of the shadow region, so the length of the shadow has a similar length. area of interest, which can contain the whole vehicle based on the shadow, can be determined. Once the ROI is determined, the taillights of the vehicle are extracted in the ROI, and the width of vehicle can be measured by the distance between the taillights. Finally, the headway distance can be measured by the estimated vehicle width and the angle of view of the camera. The proposed scheme consists of five steps: (1) road area extraction by inverse perspective mapping (IPM) [17], (2) car shadow detection, (3) ROI extraction, (4) taillights detection, and (5) inter-vehicle distance estimation. We will describe the detailed algorithms for each stage in next subsections.
A. Road Area Extraction by IPM (Inverse Perspective Mapping) First, the reduced road area is extracted by applying IPM as a predecessor work for extracting shadows from the road surface. IPM is a method of generating the result of removing the perspective effect by moving the point of the camera to the position vertically above the road, as shown in Figure 2. The image provided by the mobile application shows the road surface at the bottom of the center of the image, as shown in Figure 3-(1), except that the distance between the vehicle and the front vehicle is very close. By applying IPM to the road area, the image as shown in Figure 3-(2), which is similar to being captured from the position vertically top from the road, and in which the background area of the image has been removed, can be obtained. The length of the shadow region is relatively different depending on the distance between the vehicle and the front vehicle. This reduces the relative distance difference of the shadow region, so the length of the shadow has a similar length.
(1) The source image (2) IPM applied output image The Algorithm 1 for applying IPM is given below: The Algorithm 1 for applying IPM is given below: The function of getPerspectiveTransform() finds a 3 × 3 perspective transform matrix to transform the coordinates of src to the coordinates of dst. Suppose that src.coordinate is {(x 0 , y 0 ), (x 1 , y 1 ), . . .} and dst.coordinate is {(x 0 , y 0 ), (x 1 , y 1 ), . . .}. The perspective transform matrix T satisfies the following equation:  The function of remap() performs geometric transformation on the image by the mapping function Map_X and Map_Y. That is, dst(x, y) is determined as src(Map_X(x, y), Map_Y(x, y)) by the remap(src, dst, Map_X, Map_Y). The iteration part of the function finds the mapping function based on the perspective transform matrix. Finally, the IPM-applied dst can be obtained.

B. Car Shadow Detection
The shadow of the car can be extracted from the IPM-applied image using the color histogram. First, in the HSV (Hue, Saturation, Value) color space, only the area whose brightness (value: 0~255) is 70 or less is classified, as shown in Figure 4-(2). At this point, other objects besides the car shadow can be recognized as shadows, too. In order to find the car shadow, the following characteristics of the car shadows are used: (1) the front car is in the center of the road, (2) shadow appears at the bottom, and (3) there is no obstacle between the front car and the traveling car. Based on the features, the area that is positioned at the center bottom of the image is determined as the car shadow. Consequently, the candidate at the bottom between the two red circled candidates in Figure 4-(3) is determined as the car shadow.
The function of getPerspectiveTransform() finds a 3 × 3 perspective transform matrix to transform the coordinates of src to the coordinates of dst. Suppose that src.coordinate is { , , , , …} and dst.coordinate is { ′ , ′ , ′ , ′ , …}. The perspective transform matrix T satisfies the following equation: The function of remap() performs geometric transformation on the image by the mapping function Map_X and Map_Y. That is, dst(x, y) is determined as src(Map_X(x, y), Map_Y(x, y)) by the remap(src, dst, Map_X, Map_Y). The iteration part of the function finds the mapping function based on the perspective transform matrix. Finally, the IPM-applied dst can be obtained.

B. Car Shadow Detection
The shadow of the car can be extracted from the IPM-applied image using the color histogram. First, in the HSV (Hue, Saturation, Value) color space, only the area whose brightness (value: 0~255) is 70 or less is classified, as shown in Figure 4-(2). At this point, other objects besides the car shadow can be recognized as shadows, too. In order to find the car shadow, the following characteristics of the car shadows are used: (1) the front car is in the center of the road, (2) shadow appears at the bottom, and (3) there is no obstacle between the front car and the traveling car. Based on the features, the area that is positioned at the center bottom of the image is determined as the car shadow. Consequently, the candidate at the bottom between the two red circled candidates in Figure 4- (3) is determined as the car shadow.
(1) The IPM output (2) Histograms with the value of HSV under 70 (3) The candidate at the bottom is determined as car shadow  By applying the mapping functions used for IPM image generation inversely, the shadow area coordinates on the original image are acquired.

C. ROI (Region of Interest) Configuration
Once the car shadow area has been detected, the ROI area for detecting the taillights of the vehicle is configured. The ROI is defined as the minimum square area, which includes the entire front car in the center of the area. Since the width of the shadow is always similar to the width of the car, the ROI can be determined based on the width of the shadow. Let the width of the shadow be S. The width of ROI is set as 1.5S, and the height of ROI is set as 0.5S. Finally, the ROI, as shown in Figure 5, is defined as a square with the previously defined width and height based on the top of the shadow. By applying the mapping functions used for IPM image generation inversely, the shadow area coordinates on the original image are acquired.

C. ROI (Region of Interest) Configuration
Once the car shadow area has been detected, the ROI area for detecting the taillights of the vehicle is configured. The ROI is defined as the minimum square area, which includes the entire front car in the center of the area. Since the width of the shadow is always similar to the width of the car, the ROI can be determined based on the width of the shadow. Let the width of the shadow be S. The width of ROI is set as 1.5S, and the height of ROI is set as 0.5S. Finally, the ROI, as shown in Figure 5, is defined as a square with the previously defined width and height based on the top of the shadow.
(1) S is the width of the shadow.
(2) The width and height of the ROI are determined based on S. However, the extraction of the shadow area can fail either if the headway distance is too close, so that the shadow of the vehicle has been hidden by the current vehicle, or if there is no vehicle in front of the current vehicle. In the case of failing to extract the shadow area, a predefined ROI area is used. As shown in Figure 6, the ROI is determined by the systemically predefined road bottom width. The Algorithm 2 to configure the ROI is given below: However, the extraction of the shadow area can fail either if the headway distance is too close, so that the shadow of the vehicle has been hidden by the current vehicle, or if there is no vehicle in front of the current vehicle. In the case of failing to extract the shadow area, a predefined ROI area is used. As shown in Figure 6, the ROI is determined by the systemically predefined road bottom width. By applying the mapping functions used for IPM image generation inversely, the shadow area coordinates on the original image are acquired.

C. ROI (Region of Interest) Configuration
Once the car shadow area has been detected, the ROI area for detecting the taillights of the vehicle is configured. The ROI is defined as the minimum square area, which includes the entire front car in the center of the area. Since the width of the shadow is always similar to the width of the car, the ROI can be determined based on the width of the shadow. Let the width of the shadow be S. The width of ROI is set as 1.5S, and the height of ROI is set as 0.5S. Finally, the ROI, as shown in Figure 5, is defined as a square with the previously defined width and height based on the top of the shadow.
(1) S is the width of the shadow.
(2) The width and height of the ROI are determined based on S. However, the extraction of the shadow area can fail either if the headway distance is too close, so that the shadow of the vehicle has been hidden by the current vehicle, or if there is no vehicle in front of the current vehicle. In the case of failing to extract the shadow area, a predefined ROI area is used. As shown in Figure 6, the ROI is determined by the systemically predefined road bottom width. The Algorithm 2 to configure the ROI is given below: The Algorithm 2 to configure the ROI is given below: (a) For the HSV color space, the hue value (hue: 0 •~1 80 • ) is less than or equal to 10 • or greater than or equal to 170 • . (b) They are located symmetrically in the left and right area from the center of the ROI, respectively. (c) The size of red area is bigger than the predefined threshold value α (d) For the two symmetric red areas, the height difference is less than δ, and the slope difference is less than ε. (e) The shape of the two selected areas is almost the same and left-right symmetric.

D. Taillights Detection
After configuring the ROI area, the detection of taillights is performed. The taillights of the vehicle have common features, such as: (1) they are always symmetrically positioned at the rear left and right ends of the vehicle, and (2) they are all in red color. Thus, if all the following five conditions are satisfied, it is detected as a pair of taillights.
(a) For the HSV color space, the hue value (hue: 0°~180°) is less than or equal to 10° or greater than or equal to 170°. (b) They are located symmetrically in the left and right area from the center of the ROI, respectively. (c) The size of red area is bigger than the predefined threshold value α (d) For the two symmetric red areas, the height difference is less than δ, and the slope difference is less than ε. (e) The shape of the two selected areas is almost the same and left-right symmetric. Figure 7 shows the taillight area detected by our algorithms.
(1) The original taillight in ROI (2) Detected taillight As shown in Figure 7-(2), every red area in the ROI is regarded as a taillight candidate. For every candidate, morphological dilation and erosion have been performed to remove noise first, and then, the ROI is split into the left area and the right area by the center of the ROI. For every candidate, the steps from (b) to (e) are repeatedly performed. Suppose a candidate in the left area as red_left, and another candidate in the right area as red_right. Our algorithm decides if both are the closest to the center of the ROI, if the size of each is bigger than α, if the slope difference between two candidates As shown in Figure 7-(2), every red area in the ROI is regarded as a taillight candidate. For every candidate, morphological dilation and erosion have been performed to remove noise first, and then, the ROI is split into the left area and the right area by the center of the ROI. For every candidate, the steps from (b) to (e) are repeatedly performed. Suppose a candidate in the left area as red_left, and another candidate in the right area as red_right. Our algorithm decides if both are the closest to the center of the ROI, if the size of each is bigger than α, if the slope difference between two candidates is less than ε, and if the height difference between them is less than δ. Finally, the test for the shape similarity and symmetricity is carried out.
Suppose the left-right rotated result of red_left as red_left', as shown in Figure 8-(2). The test simply computes the difference between red_left' and red_right. The difference can be computed either by (red_left' − red_right) or by (red_right − red_left'). The smallest difference among them is adopted as the difference. If the size of the difference is less than or equal to 1/5 of red_left, then both candidates are determined as the same and symmetric.
Sensors 2018, 18, x FOR PEER REVIEW 9 of 20 is less than ε, and if the height difference between them is less than δ. Finally, the test for the shape similarity and symmetricity is carried out. Suppose the left-right rotated result of red_left as red_left', as shown in Figure 8-(2). The test simply computes the difference between red_left' and red_right. The difference can be computed either by (red_left' − red_right) or by (red_right − red_left'). The smallest difference among them is adopted as the difference. If the size of the difference is less than or equal to 1/5 of red_left, then both candidates are determined as the same and symmetric.
Consequently, a pair of red_left and red_right is detected as taillights.
(1) Split ROI into left and right space (2) Symmetry check using left/right rotation Here, since a red-colored car satisfies our proposed conditions, an exceptional processing for the red-colored car is required. If the size of candidate is larger than 1/5 of the size of ROI, such candidates are excluded from taillight candidates.

E. Inter-vehicle Distance Estimation
After taillight detection has been accomplished, the headway distance estimation to the front vehicle is performed. Since taillights are positioned at the left and right end of the vehicle, the width between taillights represents the width of the vehicle. As shown in Figure 9, suppose that the height of the image is h, the angle of view of the camera is Θ, the estimated car width is ω, the focal length of the camera is f, and the real width of the front car is W. Since the real width of the front car is different depending on the type of cars, our algorithm uses an average width (about 1.8 m) of a car for the value of W. As a result, the headway distance D is measured by the following Equations (2) and (3).   Consequently, a pair of red_left and red_right is detected as taillights.
Here, since a red-colored car satisfies our proposed conditions, an exceptional processing for the red-colored car is required. If the size of candidate is larger than 1/5 of the size of ROI, such candidates are excluded from taillight candidates.

E. Inter-vehicle Distance Estimation
After taillight detection has been accomplished, the headway distance estimation to the front vehicle is performed. Since taillights are positioned at the left and right end of the vehicle, the width between taillights represents the width of the vehicle. As shown in Figure 9, suppose that the height of the image is h, the angle of view of the camera is Θ, the estimated car width is ω, the focal length of the camera is f, and the real width of the front car is W. Since the real width of the front car is different depending on the type of cars, our algorithm uses an average width (about 1.8 m) of a car for the value of W. As a result, the headway distance D is measured by the following Equations (2) and (3). (2) Sensors 2018, 18, x FOR PEER REVIEW 9 of 20 is less than ε, and if the height difference between them is less than δ. Finally, the test for the shape similarity and symmetricity is carried out. Suppose the left-right rotated result of red_left as red_left', as shown in Figure 8-(2). The test simply computes the difference between red_left' and red_right. The difference can be computed either by (red_left' − red_right) or by (red_right − red_left'). The smallest difference among them is adopted as the difference. If the size of the difference is less than or equal to 1/5 of red_left, then both candidates are determined as the same and symmetric.
Consequently, a pair of red_left and red_right is detected as taillights.
(1) Split ROI into left and right space (2) Symmetry check using left/right rotation Here, since a red-colored car satisfies our proposed conditions, an exceptional processing for the red-colored car is required. If the size of candidate is larger than 1/5 of the size of ROI, such candidates are excluded from taillight candidates.

E. Inter-vehicle Distance Estimation
After taillight detection has been accomplished, the headway distance estimation to the front vehicle is performed. Since taillights are positioned at the left and right end of the vehicle, the width between taillights represents the width of the vehicle. As shown in Figure 9, suppose that the height of the image is h, the angle of view of the camera is Θ, the estimated car width is ω, the focal length of the camera is f, and the real width of the front car is W. Since the real width of the front car is different depending on the type of cars, our algorithm uses an average width (about 1.8 m) of a car for the value of W. As a result, the headway distance D is measured by the following Equations (2) and (3).

Driving Data Collection
The main driving data collected by a smart device consists of the current speed, mileage per second, and the headway distance to the front car. The current speed is measured as the travel distance for one second, based on the GPS position. Sathyanarayana et al. [18] showed that the current speed estimation using the GPS of a smart mobile device is as accurate as the speed value obtained from the CAN-bus of a car. The inter-vehicle distance is estimated from the images captured by the camera. The mileage per second is the Euclidean distance between the previous GPS position and the current GPS position. We denote the driving data at time T = i as D i = <V i , MD i , VD i >. V i is the current velocity (km/h), MD i is the mileage per second (meter), and VD i represents the inter-vehicle distance (meter).

Deceleration Segment Extraction and Feature Vector Generation
The next step is to extract the deceleration section from the driving data in seconds. A deceleration segment, which is denoted as DS, is a subset of sequential driving data. The initial value of DS is ∅. For T = i, DS is updated as follows: That is, D j+k = D i . For each DS output, a feature vector x i for training the learning module is computed. x i consists of seven attributes: the beginning speed of DS, the final speed of DS, the average deceleration speed, the maximal deceleration speed, the total travel distance, the total travel time, and the end inter-vehicle distance. Finally, x i is denoted as x i = <V j , V i , Avg i , Min i , TDist i , TLen i , VD i >, and each attribute value can be computed by the following Equations: Since the scale of each value in the vector differs from the others, it is finally normalized as having an average of 0 and a variation of 1.

Learning Module
There are various types of deceleration patterns, depending on the traffic conditions and driving style. The following Figure 10 shows three different types of deceleration patterns labeled Data 1, Data 2, and Data 3. All of the patterns include emergent braking. The deceleration labeled Data 1 occurred at low speed, the pattern labeled Data 2 includes sudden deceleration at the beginning, but slow deceleration around 30 km/h afterward, and the pattern of Data 3 represents risk deceleration, in which the driver felt real danger. Therefore, a technique is needed that distinguishes only the pattern of Data 3 as dangerous. In this study, we use machine learning technology to classify the risk situations that drivers perceive as dangerous. Since the proposed model should be able to be executed in real time on a smart mobile device, instead of using the deep learning technique, we use a combination of various basic machine learning techniques. For the labeled learning, we used the neural network model, which is the most typical type of labeled learning, and the random forest method, which has excellent classification performance in various situations. In addition, the k-nearest neighboring method was used as the non-labeled learning model.

A. Initial Training and Test Data Collection
First, it is necessary to obtain training data that can clearly judge whether braking is dangerous or normal. In order to achieve this, we have gathered real deceleration data that two drivers both empirically feel to be dangerous. The two drivers gathered actual driving data using our mobile application on their commutes, which included both city roads and highways. We let drivers touch the mobile device whenever they felt dangerous deceleration, and such data was collected as risk training data. Since risk deceleration does not occur often, the amount of training data was insufficient. Thus, additional experimental data, which have similar patterns to the risk training data, have been added to the training data in order to increase the reliability of our risk detection model. Details of the experimental data are described in Section 4.

B. Neural Network Module
The first machine learning model is the neural network model. We have used a multi-layer perceptron (MLP) model with two hidden layers, each of which has three nodes. It finally outputs two class labels, such that 1 means "dangerous" and 0 means "normal". For a given train set X = {(x1, c1), (x2, c2), …, (xN, cN)}, where xi is the i-th feature vector and ci is the class label ∈ 0,1 , the MLP learning function for each xt ∈ X is defined as follows: the k-th node at the j-th layer for 1 j 3: where is the i-th value at the (j−1)-th layer, is the weight at the (j−1)-th layer, and is an the bias added to the j-th layer.
k is the number of nodes at the j-th layer and p is the number of nodes at the (j−1)-th layer. j = 1 represents the first hidden layer, and j = 3 is the output layer. If j = 1, then , where is the i-th value in xt. g(x) is the activation function. The following hyperbolic tangent function has been used as the activation function: g (10) Figure 10. Various deceleration patterns, including sudden braking.
In this study, we use machine learning technology to classify the risk situations that drivers perceive as dangerous. Since the proposed model should be able to be executed in real time on a smart mobile device, instead of using the deep learning technique, we use a combination of various basic machine learning techniques. For the labeled learning, we used the neural network model, which is the most typical type of labeled learning, and the random forest method, which has excellent classification performance in various situations. In addition, the k-nearest neighboring method was used as the non-labeled learning model.

A. Initial Training and Test Data Collection
First, it is necessary to obtain training data that can clearly judge whether braking is dangerous or normal. In order to achieve this, we have gathered real deceleration data that two drivers both empirically feel to be dangerous. The two drivers gathered actual driving data using our mobile application on their commutes, which included both city roads and highways. We let drivers touch the mobile device whenever they felt dangerous deceleration, and such data was collected as risk training data. Since risk deceleration does not occur often, the amount of training data was insufficient. Thus, additional experimental data, which have similar patterns to the risk training data, have been added to the training data in order to increase the reliability of our risk detection model. Details of the experimental data are described in Section 4.

B. Neural Network Module
The first machine learning model is the neural network model. We have used a multi-layer perceptron (MLP) model with two hidden layers, each of which has three nodes. It finally outputs two class labels, such that 1 means "dangerous" and 0 means "normal". For a given train set X = {(x 1 , c 1 ), (x 2 , c 2 ), . . . , (x N , c N )}, where x i is the i-th feature vector and c i is the class label ∈ {0, 1}, the MLP learning function for each x t ∈ X is defined as follows: the k-th node at the j-th layer for 1 ≤ j ≤ 3: where Z j−1 i is the i-th value at the (j−1)-th layer, W j−1 ki is the weight at the (j−1)-th layer, and b j k is an the bias added to the j-th layer. k is the number of nodes at the j-th layer and p is the number of nodes at the (j−1)-th layer. j = 1 represents the first hidden layer, and j = 3 is the output layer. If j = 1, then Z j−1 i is the activation function. The following hyperbolic tangent function has been used as the activation function: For the k-th training sample x k , let the estimated output of x k at the output layer be O k . The output is classified into y k by the following softmax function, as in Equation (11). In addition, the error rate (E N ) between every pair of y k and c k is computed by the cross-entropy function, as in Equation (12): Consequently, the model minimizes the error rate by repeatedly updating the weights W 1 and W 2 . The error back-propagation algorithm based on the SGD (stochastic gradient descent) from the output layer to the previous layer is used to minimize the error rate.

C. Random Forest Module
Secondly, the random forest model is used to protect the risk detection model from being overfit to the training set. Since the size of the training set is not big, the number of decision trees is set to 50. Each tree has two outputs so as to match the previous neural network model. Random trees are initially established by a randomly chosen training set. The tree node split for constructing a decision tree is also performed by randomly chosen attributes. The number of randomly selected attributes for the tree node split, which is denoted as rand_features, has a significant impact on the performance of the random forest model. Thus, rand_feature is determined by the following Equation (13), as proposed by Breiman [19]. n means the total number of attributes: rand_ f eatures = int(log 2 n + 1) (13) Each node of a decision tree is split into a binary tree. Let RF = {f 1 , f 2 , . . . , f l } be a subset of randomly selected features. For each f i , it finds a reference value among all the values of f i that minimizes the entropy between the reference value and all of the other values. Suppose that a selected reference value for f i is v i . For a given training sample data, the entropy by v i is determined by the following entropy function E R : L = a subset of data that has values lower than v i ; U = a subset of data that has values equal or greater than v i ; eL = −(P(class = 0|L) log 2 P(class = 0|L) + P(class = 1|L) log 2 P(class = 1|L)) ; eU = −(P(class = 0|U) log 2 P(class = 0|U) + P(class = 1|U) log 2 P(class = 1|U)) ; The node is finally split by the feature that has the smallest entropy. The tree node split is also repeated until no further split occurs. Once a random forest has been established, the final output is determined as the most frequent decision of the decision trees.

D. Clustering Module
In order to make the proposed model work effectively even for untrained risk situations, a clustering model, which is a typical unsupervised learning method, is used as well. Our training dataset consists of seven attributes. In order to increase the accuracy of clustering, the two main attributes that have the greatest effect on the classification are extracted, and the clustering is performed based on the principal attributes. The principal components were extracted from the histogram analysis of the "dangerous" and "normal" conditions for each attribute. We have simply compared the difference in average values between both groups for each attribute; the maximum deceleration has about 47% difference, the average deceleration has about 29% difference, the inter-vehicle distance has about 18% difference, the end speed has about 12% difference, and the remainders have about 5% difference. Thus, the "maximum deceleration" and "average deceleration" were extracted as the main components. Finally, a kNN (k Nearest Neighbors) algorithm is used for the classification based on the main components for the following two reasons. (1) We have the training dataset that has been already classified.
(2) The performance of k-means clustering, which is another representative clustering algorithm, deeply depends on the initial cluster heads that are randomly selected. The number of k neighbors is set to 5. The Euclidean distance has been used to compute the distance between two points. For input data, classification is computed from a simple majority vote of the nearest neighbors of each input point: the data class that has the most representatives within the nearest neighbors of the input point is assigned to the input data.
In this way, the initial learning module based on the three different machine learning algorithms is established.

A. Making the Final Decision
Based on the initial learning module, the decision module makes the final decision on new driving data in real-time. The decision classifies braking as either "danger", "suspicious", or "normal". During driving, the driving data management module extracts deceleration segments, and every deceleration segment is given to the decision module. We have experimented with two decision algorithms named DM all and DM 2 in this study. For each deceleration segment, DM all works as follows: If all models decide as danger, the final decision is "danger", If all models decide as normal, the final decision is "normal", otherwise, the final decision is "suspicious".
In contrast, DM 2 works as follows: If two or more models decide as danger, the final decision is "danger", If all models decide as normal, the final decision is "normal", otherwise, the final decision is "suspicious".
From the conclusion, it is experimentally proven that the performance of DM 2 is better than that of DM all , so DM 2 has been adopted as our main decision rule. The performance analysis of the two methods will be described in detail in Section 4. Finally, a warning message is transmitted to the traffic server if and only if the final decision is "dangerous".

B. Enhancing the Risk-Detecting Model
The deceleration data through which final decisions have been made are used as new training data for updating and enhancing the risk situation detection model. As driver-specific deceleration data is continually added to the common initial risk-detection model, the reliability of the initial model can be continuously enhanced, and the model is updated to a driver-customized model. Three types of enhancement mechanisms, denoted as EH all , EH 2 , and EH w , have been considered.
• EH all only uses data for which all of the learning models made the same output. That is, data that all of the models output as "dangerous" is used as "dangerous" training data. Likewise, data that all of the models output "normal" is used as "normal" training data.
• EH 2 follows the final decision of the above decision module DM 2 . That is, the data for which the final decision was "dangerous" is used as "dangerous" training data, and the data for which the final decision was "normal" is used as "normal" training data. • EH w assigns different weights according to the decisions of the three models. It is the same as EH 2 , except that the data that all of the models output as "dangerous" are doubly copied and used for training data. If all of the models decide that some data are "dangerous", the risk dataset is then slightly more enhanced, because the amount of risk training data has increased.
The performance analysis results according to the enhancement mechanisms are described in detail in Section 4. Since the size of the general data compared to the risk data is so large, some of the general data that are randomly extracted from the general data are used as training data. The new training data are added to the existing training data. The risk detection enhancement works periodically by the learning module.

Simulated Performance
In this section, we describe the simulated results of the proposed traffic risk detection model through real driving on Seoul city roadways and Gyeongbu expressway. First, we analyze the performance of the proposed headway distance estimation against three aspects: real-time processing performance, vehicle detection, and the accuracy of the distance estimation. Then, we analyze the accuracy of the risk detection according to the proposed decision models and the proposed enhancement methods. Figure 11 shows the implemented result by our mobile application in real time.

Simulated Performance
In this section, we describe the simulated results of the proposed traffic risk detection model through real driving on Seoul city roadways and Gyeongbu expressway. First, we analyze the performance of the proposed headway distance estimation against three aspects: real-time processing performance, vehicle detection, and the accuracy of the distance estimation. Then, we analyze the accuracy of the risk detection according to the proposed decision models and the proposed enhancement methods. Figure 11 shows the implemented result by our mobile application in real time.

The Performance of the Headway Distance Estimation
(1) The detection of shadow and ROI (2) The estimated headway distance to the front car Figure 11. The implemented results of the proposed model.
We have compared the throughput rate of image processing for three types of smart mobile devices. Table 1 shows the performance for the IPM image generation time, the shadow area detection time, the taillight detection time, and the number of image frames processed per second according to three different Samsung Galaxy series J7, A8, and S7. S7 showed the best performance; it took about 7 ms for the IMP image creation, and less than 0.1 ms for the shadow area detection, on all types of mobile devices. The taillight detection took the longest time; it took about 17 ms on all of the models. In addition, at least 37 image frames per second have been processed on all of the models.  We have compared the throughput rate of image processing for three types of smart mobile devices. Table 1 shows the performance for the IPM image generation time, the shadow area detection time, the taillight detection time, and the number of image frames processed per second according to three different Samsung Galaxy series J7, A8, and S7. S7 showed the best performance; it took about 7 ms for the IMP image creation, and less than 0.1 ms for the shadow area detection, on all types of mobile devices. The taillight detection took the longest time; it took about 17 ms on all of the models. In addition, at least 37 image frames per second have been processed on all of the models. Secondly, we have analyzed the vehicle detection rate according to various driving environments. The driving environments have been classified into daytime urban roads, daytime expressways, and night in consideration of illuminance and speed. Table 2 summarizes the vehicle detection rate according to the driving environment. The target frame is defined as a frame that includes the shadow area of the front vehicle. The vehicle detection rate is measured with the number of frames where the detection of the front vehicle succeeded among the target frames. The detection rate during the daytime is more than 90%, and it is relatively higher on urban roadways. On the other hand, the detection rate at night was very low, less than 50%. As shown in Table 2, the shadow detection at night is still high, but the taillight detection rate is very low at night, since the taillight was not detected in red. Further research is needed to increase the taillight detection rate at night. Finally, the accuracy of the headway distance to the front vehicle has been analyzed. To achieve this, a laser-based distance-measuring sensor (LIDAR-Lite V3) has been additionally equipped to the experiment vehicle. In order to obtain more accurate data, two types of test vehicles (sedan and SUV) have been used in our experiment. The headway distance has been obtained by following the front experimental vehicle. The error between the laser sensor value and the estimated value of the proposed model has been analyzed. Table 3 shows the results of the error values for the two types of vehicles. As shown in Table 3, the distance accuracy is more than 96% within 15 m, but the accuracy decreases a little bit over long distances. The average error of the proposed model is 4.35%, whereas the average error of the laser sensor is 1.8%. However, the error rate is less than 5%, so we can conclude that it can be applied to the real situation practically.

The Performance of the Traffic Risk Detection
Now, in this section, we describe the simulated results on the accuracy of the proposed traffic risk detection model in detail. We explain the experimental dataset first, and analyze the accuracy according to the proposed decision models and the enhancement methods.

Experimental Data
The training data and test data are given from the actual driving data generated by two drivers. They commuted on Seoul city roads and the Gyeongbu expressway to collect these data. As mentioned before, we have collected the real dangerous deceleration data that both drivers empirically felt to be dangerous. Initially, 40 actual dangerous data were obtained, and 20 experimental data that had similar patterns to the real dangerous data were added. Among them, 26 dangerous training data were used to build up the initial learning module of our system, and the remainders were used as test data. Based on the initial learning module, our system has continuously collected new training and test datasets by detecting dangerous situations during actual driving. The Samsung Galaxy S7 smart phone was used in this experiment. For each deceleration segment, it takes 0.2 s until the final decision. The newly obtained data have been used as training data to reinforce the learning module, or have been used as test data. Two types of test datasets have been used. Type_1 test data are similar to the training data, but Type_2 test data include certain risk situations that are not similar to the training set. Table 4 shows the size of the experimental data set used in the simulation. As mentioned above, the new input data with the final decision confirmed by our system is recycled as training data to reinforce the learning model. Table 5 shows the training data used for the three types of reinforcement algorithms. Three steps of enhancement have been performed in total, and the table shows the datasets used for each. We have defined two types of scores to analyze the accuracy of our risk detection model. The first indicator is a risk score denoted as RScore, which measures only the accuracy of the risk data. Thus, it shows how well our model detects a danger situation. For each deceleration segment DS i , suppose that class(DS i ) denotes the given label of DS i , and predict(DS i ) means the label estimated by our model. Therefore, for all of the deceleration segment DS values, the RScore is defined as follows: The second indicator is a total score, which is denoted as TScore. TScore measures the accuracy of the proposed model for both general and risk data. TScore is defined as follows: Consequently, we analyze the scores of the two proposed decision models for initial training data and the two types of test data, and compare the performance with existing learning models. We also compare the performances of the proposed three types of enhancement models. Since the simulated results are slightly different every time the learning models work with the given training data, we have analyzed the average results of 10 different experiments. Table 6 shows both the RScore and TScore of the initial risk detection model for the initial training data. Among the existing learning models, the random forest model shows the best accuracy for the training data. Among the proposed decision models, DM 2 is more accurate than DM all , and DM 2 shows the next best in the accuracy for the training data. Table 7 summarizes the performance of the initial risk detection model for the two types of test data. Among the existing learning models, the neural network model shows the best accuracy for the test data, in which DM 2 is still shown to be superior to DM all . Therefore, DM 2 offers the best overall performance. As shown in the above experimental results, each learning model shows a different performance depending on the situation. While random forest has the best performance for the training data, it has the lowest performance for the test data used in our experiments. The neural network has excellent performance for the test data rather than the training data. kNN shows moderate performance across the training and test data. We believe that it is appropriate to use all three models, because all types of sudden braking, whether those are similar to the training data or not, can occur in the actual situation.
Next, we describe the performance analysis results according to various enhancement methods. Each learning model has been updated by three enhancement techniques, respectively. Table 8 summarizes the RScore of each updated risk detection model for its new training data. The random forest and kNN show better accuracy for the training data, even in the enhanced models, than the neural network model. DM 2 offers the next-best performance in the enhanced models as well. The TScore of the updated risk detection model for its new training data is more than 0.99 for all of the cases.  Figure 12 shows the RScore of each risk detection model updated by each enhancement method for two types of test data. Since the three learning modules produce different performances depending on the enhancement techniques and the types of test data, it shows the RScore of our proposed two decision models compared with the minimum and maximum accuracy values produced by the three learning modules. As shown in Figure 12, the RScore of EH w is the best, as well as the same as the maximum value among the RScores of the three learning modules for all kinds of test data. In contrast, the RScore of EH all is the worst. In the EH all method, the risk detection model is updated with only data that all of the learning modules deemed to be a "risk". That is, only limited learning data on risk situations are used for the risk detection reinforcement. Therefore, the detection rate is inevitably low. In both EH 2 and EH w , since all of the data that at least two learning modules have judged as a "risk" are used to reinforce the risk situation, more diverse risk situations are reflected to the updated model. Furthermore, in EH w , if all of the models decide that the data is a "risk", the data is duplicated and used as new training data, so that the training data for the risk situation increases, and EH w has a higher accuracy for the detection of dangerous situations than EH 2 .
Sensors 2018, 18, x FOR PEER REVIEW 18 of 20 Figure 12. RScore of the enhanced models for the initial training data and test data. Figure 13 shows the TScore of the enhanced models for two types of test data, including the initial training data. The TScore indicates the accuracy of both the risk detection rate and the general situation detection rate. For the initial training data and the Type_1 test data, every type of enhanced model shows a similar accuracy of more than 0.98. For Type_2 test data, only models enhanced with EHw still show an accuracy of 0.98 or better, while the remainder is somewhat lower. It is determined that the low RScore for Type_2 also affects the TScore. Nevertheless, the TScore is 0.96 or more for all of the enhancement models, and the TScore deviation according to enhancement techniques is less than 0.02. Therefore, the performances of all of the reinforcement models are similar in terms of TScore. Figure 12. RScore of the enhanced models for the initial training data and test data.
According to the experimental results, the initial DM 2 model has an RScore of 0.95 for Type_1 test data, but the RScore of the DM 2 model enhanced by EH w is 0.99, which is close to the accuracy of 1. The RScore for the initial training data has been improved to 1 as well. The initial DM 2 model has an RScore of about 0.71 for Type_2 test data, while the RScore of the DM 2 model enhanced with EH w is 0.89. Thus, the accuracy increased by about 0.18 points. This score is still unsatisfactory, but it is expected to further improve as learning data on risk situations is continuously added. As a result, we can conclude that our proposed enhancing module increases the detection rate for various dangerous situations, including the initial training data. Figure 13 shows the TScore of the enhanced models for two types of test data, including the initial training data. The TScore indicates the accuracy of both the risk detection rate and the general situation detection rate. For the initial training data and the Type_1 test data, every type of enhanced model shows a similar accuracy of more than 0.98. For Type_2 test data, only models enhanced with EH w still show an accuracy of 0.98 or better, while the remainder is somewhat lower. It is determined that the low RScore for Type_2 also affects the TScore. Nevertheless, the TScore is 0.96 or more for all of the enhancement models, and the TScore deviation according to enhancement techniques is less than 0.02. Therefore, the performances of all of the reinforcement models are similar in terms of TScore. detection rate. For the initial training data and the Type_1 test data, every type of enhanced model shows a similar accuracy of more than 0.98. For Type_2 test data, only models enhanced with EHw still show an accuracy of 0.98 or better, while the remainder is somewhat lower. It is determined that the low RScore for Type_2 also affects the TScore. Nevertheless, the TScore is 0.96 or more for all of the enhancement models, and the TScore deviation according to enhancement techniques is less than 0.02. Therefore, the performances of all of the reinforcement models are similar in terms of TScore. Figure 13. TScore of the enhanced models for initial training data and test data. Figure 13. TScore of the enhanced models for initial training data and test data.
As a result, we can conclude that the DM 2 decision model enhanced with EH w shows the best performance for risk situation detection from our experiments. Risk situations occur very rarely under normal driving conditions, so there is overwhelmingly more data on general situations than risk situations, even if some selected general situations are used for learning. Therefore, EH w , which can train more dangerous situations, even though the data used is redundant, is more suitable for detecting dangerous situations than EH 2 .

Conclusions and Future Work
We have proposed a real-time traffic risk detection model based on a smart mobile device. Our model monitors driving behaviors using only the sensors of a smart mobile device, and extracts deceleration segments. We have also proposed a practical vehicle detection and headway distance estimation scheme for risk detection. Since emergent braking happens almost inevitably and exclusively in risk situations, our model detects risk situations by analyzing deceleration patterns using machine learning algorithms. We have used actual driving data obtained from two drivers by driving on Seoul city roadways and the Gyengbu expressway for three months in order to develop normal and dangerous driving deceleration patterns. Three different machine learning algorithms, which are neural network, random forest, and clustering, were used in parallel to build our learning module. We have also suggested two decision algorithms and three enhancement algorithms to improve our risk detection model continuously. Finally, we have analyzed the performance of the headway distance estimation and the risk detection accuracy of our proposed decision models and enhancing models. From our simulated results, the vehicle detection rate on urban roads at daytime is more than 92%, and the detection rate on highways is still more than 90%. The accuracy of the headway distance estimation is more than 96% within 15 m. The traffic risk detection model applied with decision model DM 2 and enhanced with EH w shows the best performance on both the training data and test data. The model shows an accuracy of 0.99 for risk situations such as the training data, and shows a detection accuracy of 0.89 for unfamiliar risk situations.
We need further research to improve the detection rate at night. In addition, we plan to extend our work to a model that can reflect not only deceleration patterns but also various driving behaviors, such as steering, handling, and emergent acceleration for risk situation detection.