# Comparative Analysis of Classifiers for Classification of Emergency Braking of Road Motor Vehicles

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Methods

#### 2.1. Quality Metrics

#### 2.2. Cross-Validation

#### 2.3. Threshold Classifier

#### 2.4. K-Nearest Neighbors

#### 2.5. Support Vector Machine

_{i}, y

_{i}), where x

_{i}∈ ${\mathbb{R}}^{n}$ and y

_{i}∈ $\mathbb{R}$ are the i-th input and output samples, respectively. SVM constructs a classifier, which can be represented by a function:

_{k}are support vectors, α

_{i}are positive real numbers, while b (bias) is a real number. The kernel K can be used in various forms: ${x}_{k}^{T}x$ (linear kernel), ${(\gamma {x}_{k}^{T}x+r)}^{n}$ (polynomial kernel of degree n), ${e}^{-\gamma {\Vert x-{x}^{\prime}\Vert}^{2}}\left(\mathrm{radial}\text{}\mathrm{basis}\text{}\mathrm{function}\text{}\mathrm{kernel}\right)$, or $\mathrm{tanh}\left[\gamma {x}_{k}^{T}x+r\right]$ (sigmoid kernel), where kernel parameters $\gamma =\frac{1}{2{\sigma}^{2}}$ and $r$ are real constants.

#### 2.6. Extreme Gradient Boosting

_{k}corresponds to independent tree structure and leaf weights, while x

_{i}is a testing object. F represents a testing space of all CART. For learning the set of functions, the following objective is used:

_{t}(x

_{i}) to our objective function.

_{j}= {i|q(x

_{i}) = j} as an instance set of leaf j for a given tree structure q(x), the objective function can be rewritten as follows:

_{j}of leaf j will be

_{L}and I

_{R}are the instance sets of the left and right nodes after the split, I = I

_{L}∪ I

_{R}, and $\gamma $ is the regularization on the additional leaves. It is important to note that—besides the regularization technique—shrinkage and feature subsampling are also used in XGBoost [22].

#### 2.7. Genetic Algorithms

_{1}, x

_{2}, …, x

_{n}) where each coordinate x

_{i}of the vector—the value of a parameter to be optimized—represents a genetic allele. The objective of applying GA is to find such a chromosome that results in an optimal value of a given target—fitness function F. It is important to note that the design of fitness function F is domain-specific, and depends on the nature of the problem being solved. In GA, the quest for the best chromosome (solution), i.e., the chromosome that yields an optimal value of F, consists of the following main steps:

- Step 0
- Creating the initial population of randomly generated chromosomes;
- Step 1
- Evaluating the fitness of chromosomes in the population;
- Step 2
- Checking the termination criteria: good enough fitness value (of the best chromosome in the population), too long runtime, or too many generations. GA terminates if one of the criteria is satisfied;
- Step 3
- Selection: selecting the mating pool of chromosomes. The size of the mating pool is a fraction (i.e., 10%) of the overall size of the population, and the selection of the chromosomes in the mating pool is fitness-proportional (roulette-wheel, tournament, elitism, etc.);
- Step 4
- Reproduction: implementing crossover by swapping random fragments of randomly selected pairs (parents) of chromosomes from the mating pool. Crossover produces pairs of offspring chromosomes that are inserted into the newly growing population; and
- Step 5
- Mutation: random gene(s) of newly generated offspring chromosomes are randomly modified with a given probability.

Generate initial population;Evaluate population;While not(Termination Criteria)doSelection;// Creating the mating pool of surviving chromosomesReproduction;// Crossing over a randomly selected pair of survivedchromosomesMutation;// Mutating the newly produced (by crossover) offspringEvaluate population;

## 3. Proposals

## 4. Methodology

#### 4.1. Acquiring Data

#### 4.2. Data Analysis and Feature Extraction

#### 4.3. Building Classifiers

#### 4.4. Quality Estimation and Applying of GA

## 5. Experimental Results

## 6. Discussion

## 7. Conclusions

## Acknowledgments

## Author Contributions

## Conflicts of Interest

## References

- Laukkonen, J. 13 Advanced Driver Assistance Systems. Available online: https://www.lifewire.com/advanced-driver-assistance-systems-534859 (accessed on 8 November 2017).
- Coelingh, E.; Eidehall, A.; Bengtsson, M. Collision Warning with Full Auto Brake and Pedestrian Detection—A Practical Example of Automatic Emergency Braking. In Proceedings of the 13th International IEEE Conference on Intelligent Transportation Systems, Funchal, Portugal, 19–22 September 2010; pp. 155–160. [Google Scholar] [CrossRef]
- Coelingh, E.; Jakobsson, L.; Lind, H.; Lindman, M. Collision Warning with Auto Brake—A Real-Life Safety Perspective. In Proceedings of the 20th International Technical Conference on the Enhanced Safety of Vehicles (ESV), Lyon, France, 18–21 June 2007. [Google Scholar]
- Kusano, K.D.; Gabler, H.C. Safety Benefits of Forward Collision Warning, Brake Assist, and Autonomous Braking Systems in Rear-End Collisions. IEEE Trans. Intell. Transp. Syst.
**2012**, 13, 1546–1555. [Google Scholar] [CrossRef] - Fancher, P.; Bareket, Z.; Ervin, R. Human-Centered Design of an Acc-With Braking and Forward-Crash-Warning System. Int. J. Veh. Mech. Mobil.
**2010**, 36, 203–223. [Google Scholar] - Wilde, G.J.S. The theory of risk homeostasis: Implications for safety and health. Risk Anal.
**1982**, 2, 209–225. [Google Scholar] [CrossRef] - Haufe, S.; Treder, M.S.; Gugler, M.F.; Sagebaum, M.; Curio, G.; Blankertz, B. EEG potentials predict upcoming emergency brakings during simulated driving. J. Neural Eng.
**2011**, 8, 1–11. [Google Scholar] [CrossRef] [PubMed] - Haufe, S.; Kim, J.; Kim, I.H.; Treder, M.S.; Sonnleitner, A.; Schrauf, M.; Curio, G.; Blankertz, B. Electrophysiology-based detection of emergency braking intention in real-world driving. J. Neural Eng.
**2014**, 11, 056011. [Google Scholar] [CrossRef] [PubMed] - Kim, I.H.; Kim, J.W.; Haufe, S.; Lee, S.W. Detection of braking intention in diverse situations during simulated driving based on EEG feature combination. J. Neural Eng.
**2015**, 12, 016001. [Google Scholar] [CrossRef] [PubMed] - Mccall, J.C.; Trivedi, M.M. Human Behavior Based Predictive Brake Assistance. In Proceedings of the 2006 IEEE Intelligent Vehicles Symposium, Tokyo, Japan, 13–15 June 2006; pp. 8–12. [Google Scholar] [CrossRef]
- Kassaagi, M.; Brissart, G.; Popieul, J.-C. A study on driver behavior during braking on open road. In Proceedings of the 18th International Technical Conference on the Enhanced Safety of Vehicles (ESV), Nagoya, Japan, 19–22 May 2003. [Google Scholar]
- Kiesewetter, W.; Klinkner, W.; Reichelt, W.; Steiner, M. Der neue Brake-Assist von Mercedes-Benz. Automobiltech. Z.
**1997**, 99, 330. [Google Scholar] - Malta, L.; Miyajima, C.; Takeda, K. A Study of Driver Behavior under Potential Threats in Vehicle Traffic. IEEE Trans. Intell. Transp. Syst.
**2009**, 10, 201–210. [Google Scholar] [CrossRef] - Podusenko, A.; Nikulin, V.; Tanev, I.; Shimohara, K. Cause and Effect Relationship between the Dynamics of Accelerator and Brake Pedals during Emergency Braking. In Proceedings of the FAST-zero 2017, Nara, Japan, 22–28 September 2017. [Google Scholar]
- Forum-8 Drive Simulator. Available online: http://www.forum8.co.jp/english/uc-win/road-drive-e.htm (accessed on 18 November 2017).
- Podusenko, A.; Nikulin, V.; Tanev, I.; Shimohara, K. Prediction of Emergency Braking Based on the Pattern of Lifting Motion of Accelerator Pedal; SICE: Osaka, Japan, 2017. [Google Scholar]
- Powers, D.M.W. Evaluation: From Precision, Recall and F-Measure to ROC. Inf. Mark. Correl. J. Mach. Learn. Technol.
**2011**, 2, 37–63. [Google Scholar] - Kohavi, F. A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection. In Proceedings of the 14th International Joint Conference on Artificial Intelligence, Montreal, QC, Canada, 20–25 August 1995. [Google Scholar]
- Refaeilzadeh, P.; Tang, L.; Liu, H. Cross-validation. In Encyclopedia of Database Systems; Springer: New York, NY, USA, 2009; pp. 532–538. [Google Scholar]
- Dasarathy, B.V. Nearest Neighbor (NN) Norms: NN Pattern Classification Techniques; IEEE Computer Society: Washington, DC, USA, 1991; ISBN 0-8186-8930-7. [Google Scholar]
- Suykens, J.; Vandewalle, J. Least Squares Support Vector Machine Classifiers. Neural Process. Lett.
**1999**, 9, 293. [Google Scholar] [CrossRef] - Chen, T.; Guestrin, C. Xgboost: A Scalable Tree Boosting System. arXiv, 2016; arXiv:1603.02754. [Google Scholar]
- Adam-Bourdarios, C.; Cowan, G.; Germain-Renaud, C.; Guyon, I.; Kégl, B.; Rousseau, D. The Higgs Machine Learning Challenge. J. Phys. Conf. Ser.
**2015**. [Google Scholar] [CrossRef] - Phoboo, A.E. Machine Learning wins the Higgs Challenge. ATLAS News, 20 November 2014. [Google Scholar]
- Lewis, R.J. An Introduction to Classification and Regression Tree (CART) Analysis. In Proceedings of the Annual Meeting of the Society for Academic Emergency Medicine, San Francisco, CA, USA, 22–25 May 2000. [Google Scholar]
- Goldberg, E.; Holland, J.H. Genetic Algorithms and Machine Learning. Mach. Learn.
**1988**, 3, 95–99. [Google Scholar] [CrossRef] - Holland, J.H. Adaptation in Natural and Artificial Systems, Reprint edition; The MIT Press; Bradford Book: Cambridge, MA, USA, 1992. [Google Scholar]
- Holland, J.H. Hidden Order: How Adaptation Builds Complexity; Basic Books: New York, NY, USA, 1996. [Google Scholar]
- Pedregosa, F.; Varoquaux, G.; Gramfort, A.; Michel, V.; Thirion, B.; Grisel, O.; Blondel, M.; Prettenhofer, P.; Weiss, R.; Dubourg, V.; et al. Scikit-Learn: Machine Learning in Python. JMLR
**2011**, 12, 2825–2830. [Google Scholar] - Buitinck, L.; Louppe, G.; Blondel, M.; Pedregosa, F.; Mueller, A.; Grisel, O.; Niculae, V.; Prettenhofer, P.; Gramfort, A.; Grobler, J.; et al. API Design for Machine Learning Software: Experiences from the Scikit-Learn Project. arXiv, 2013; arXiv:1309.0238. [Google Scholar]
- XGBoost Package. Available online: https://github.com/dmlc/xgboost (accessed on 18 November 2017).
- Podusenko, A. Classifiers Implementation of Emergency Braking Classifier in Python. 2017. Available online: http://isd-si.doshisha.ac.jp/a.podusenko/research (accessed on 18 November 2017).

**Figure 1.**Typical dynamics of accelerator and brake pedals during (a) accelerating, (b) cruising, (c) slowing down (e.g., approaching corner), (d) normal braking (e.g., approaching a stop sign), and (e) emergency braking, respectively.

**Figure 3.**Typical dynamics of accelerator and brake pedals during emergency braking. The braking behavior of the driver could be decomposed into the following three actions: lifting the accelerator (a), moving the right leg from accelerator to the brake pedal (b), and pressing the brake pedal to its (almost) maximum position (c). Depending on physical and cognitive condition of the driver, the corresponding time lags of these actions might significantly vary.

**Figure 6.**Examples of regularization parameter influence in the case of the two-dimensional classification problem. Red and blue points are training samples. The double arrow is hyperplane margin.

**Figure 7.**Breakdown of the time lag between lifting the accelerator and pressing the brake pedal completely, obtained from 383 events of emergency braking.

**Figure 8.**Relationship between the average rate of lifting the accelerator and the position of the pressed brake pedal.

**Figure 9.**Relationship between the maximum rate of lifting the accelerator and maximum rate of pressing the brake pedal.

**Figure 11.**Fitness convergence characteristics obtained from 100 independent runs of GA employed for evolution of optimal hyperparameters of SVM (

**a**) and XGBoost (

**b**), respectively. The red line indicated the average (over 100 runs) of the fitness value.

**Figure 12.**Sample dynamics of accelerator and brake pedals in two cases of emergency braking: without (

**a**) and with (

**b**) automated braking activated by the driver-supporting agent.

True Condition | |||
---|---|---|---|

Positive | Negative | ||

Predicted condition | Positive | TP | FP |

Negative | FN | TN |

Track | Length | Road Conditions | Traffic Conditions |
---|---|---|---|

Highway | 1 km | Dry | Moderate (high-speed) traffic |

Countryside road | 3 km | Both dry and wet | Empty road (no traffic) |

City road | 5 km | Both dry and wet | Dense (low-speed) traffic |

Experiment | Range of the Position of Accelerator Pedal | Sampling Frequency, Hz | Requirements | Number of Data Samples |
---|---|---|---|---|

Normal driving | [0 … 100] | 25 | Emergency braking is not allowed | 529 |

Emergency braking | [0 … 100] | 25 | Audible signal prompts the drivers to apply emergency braking | 775 |

Allele (Hyperparameter) | Interval of Discretization | Range | Meaning |
---|---|---|---|

C | 0.002 | [0, 100] | Penalty parameter C of the error term |

kernel | – | {linear, poly, rbf, sigmoid} | Specifies the kernel type to be used in SVM |

degree | 1 | [1, 5] | Degree of the polynomial kernel function. Ignored by all other kernels |

gamma | 0.1 | [0.01, 100] | Kernel coefficient for rbf, poly, and sigmoid |

coef0 | 0.1 | [0.01, 100] | Independent term in kernel function. Significant in poly and sigmoid |

shrinking | – | {True, False} | Whether to use shrinking heuristic |

**Table 5.**Content of chromosome of GA applied for optimizing the values of hyperparameters of XGBoost.

Allele (Hyperparameter) | Interval of Discretization | Range | Meaning |
---|---|---|---|

eta | 0.002 | [0, 1] | Step size shrinkage. Controls the learning rate in update and prevents overfitting |

gamma | 0.1 | [0, 100] | Minimum loss reduction required to make a node split. Split happens when the resulting split gives a positive reduction in the loss function. |

max_depth | 1 | [1, 20] | The maximum depth of a tree |

min_child_weight | 1 | [1, 100] | Minimum sum of weights of all observations required in a child |

subsample | 0.002 | (0.001, 1) | Subsample ratio of the training instance |

colsample_bytree | 0.001 | (0.5, 1) | Subsample ratio of columns when constructing each tree |

n_estimators | 1 | [100, 500] | The number of boosting stages to perform |

Parameter | Value |
---|---|

Genotype | Each classifier has its own set of parameters (as shown in Table 4 and Table 5) encoded in the genotype. In addition, a fixed combination of features (pertinent to the dynamics of the accelerator pedal) is also incorporated into the genotype: {mP, aR}, {mP, mR}, {mR, aR}, {mP, aR, mR}, {mP, aR, mR, mP*aR}, {mP, aR, mR, mP*mR}, {mP, aR, mR, mR*aR} |

Population size | 40 individuals |

Selection | Binary Tournament |

Selection ratio | 10% |

Elite | Best 2 individuals |

Crossover | Single-point |

Mutation | Single-point |

Mutation ratio | 5% |

Fitness value | F-score |

Termination criteria | (#Generations > 100) or (Fitness Value = 100%) |

Metric | Classifier Based on mR Feature | Classifier Based on aR Feature | ||
---|---|---|---|---|

Training Set | Testing Set | Training Set | Testing Set | |

Accuracy | 0.769 | 0.791 | 0.888 | 0.873 |

Precision | 1.000 | 1.000 | 0.964 | 0.980 |

Recall | 0.461 | 0.513 | 0.671 | 0.610 |

F-score | 0.631 | 0.677 | 0.791 | 0.752 |

Metric | 5-NN Method | 23-NN Method | ||
---|---|---|---|---|

Training Set | Testing Set | Training Set | Testing Set | |

Accuracy | 0.944 | 0.938 | 0.926 | 0.95 |

Precision | 0.937 | 0.867 | 0.918 | 0.896 |

Recall | 0.933 | 0.951 | 0.908 | 0.951 |

F-score | 0.935 | 0.907 | 0.913 | 0.923 |

Allele (Hyperparameter) | Value Obtained via GA |
---|---|

C | 0.802 |

kernel | rbf |

gamma | 0.8 |

coef0 | 9.0 |

shrinkage | True |

features | {mP, aR, mR} |

Allele (Hyperparameter) | Optimal Value Obtained via GA |
---|---|

eta | 0.374 |

gamma | 6.6 |

max_depth | 3 |

min_child_weight | 2 |

subsample | 0.454546 |

colsample_bytree | 0.624 |

n_estimators | 96 |

features | {mP, aR, mR, mP*aR} |

Metric | Default SVM Classifier | Evolved SVM Classifier | ||
---|---|---|---|---|

Training Set | Testing Set | Training Set | Testing Set | |

Accuracy | 0.9348 | 0.95 | 0.9396 | 0.95 |

Precision | 0.9479 | 0.9058 | 0.9465 | 0.9058 |

Recall | 0.8971 | 0.939 | 0.9105 | 0.939 |

F-score | 0.9218 | 0.9221 | 0.9281 | 0.9221 |

Metric | Default XGBoost | Evolved XGBoost | ||
---|---|---|---|---|

Training Set | Testing Set | Training Set | Testing Set | |

Accuracy | 0.9502 | 0.9461 | 0.953 | 0.9538 |

Precision | 0.954 | 0.8953 | 0.9461 | 0.917 |

Recall | 0.9284 | 0.939 | 0.944 | 0.939 |

F-score | 0.941 | 0.917 | 0.9451 | 0.9277 |

Metric | 23-NN Method | Default (and Evolved) SVM | Evolved XGBoost |
---|---|---|---|

Accuracy | 0.95 | 0.95 | 0.9538 |

Precision | 0.896 | 0.9058 | 0.917 |

Recall | 0.951 | 0.939 | 0.939 |

F-score | 0.923 | 0.9221 | 0.9277 |

Speed of the Car | Distance between Two Cars, m | Reduction of the Distance to the Following Car as a Result of Eventual False Positive Emergency Braking of 200 ms, m | ||
---|---|---|---|---|

km/h | m/s | For 3 s Interval between Cars (Marginal) | For 6 s Interval between Cars (Good) | |

36 | 10 | 30 | 60 | 2 |

54 | 15 | 45 | 90 | 3 |

72 | 20 | 60 | 180 | 4 |

90 | 25 | 75 | 150 | 5 |

108 | 30 | 90 | 180 | 6 |

126 | 35 | 105 | 210 | 7 |

© 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Podusenko, A.; Nikulin, V.; Tanev, I.; Shimohara, K. Comparative Analysis of Classifiers for Classification of Emergency Braking of Road Motor Vehicles. *Algorithms* **2017**, *10*, 129.
https://doi.org/10.3390/a10040129

**AMA Style**

Podusenko A, Nikulin V, Tanev I, Shimohara K. Comparative Analysis of Classifiers for Classification of Emergency Braking of Road Motor Vehicles. *Algorithms*. 2017; 10(4):129.
https://doi.org/10.3390/a10040129

**Chicago/Turabian Style**

Podusenko, Albert, Vsevolod Nikulin, Ivan Tanev, and Katsunori Shimohara. 2017. "Comparative Analysis of Classifiers for Classification of Emergency Braking of Road Motor Vehicles" *Algorithms* 10, no. 4: 129.
https://doi.org/10.3390/a10040129