# Research on an Intelligent Behavior Evaluation System for Unmanned Ground Vehicles

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Test Content Design

#### 2.1. Test Content Scenario Design

#### 2.2. Layer Test Content Design

## 3. Test Environment Design

#### 3.1. A Test Environment Model Establishment

#### 3.2. Test Layer Environment Design

## 4. Intelligent Behavior Evaluation Method for UGV

#### 4.1. Establishment of Index Evaluation System

#### 4.2. EAHP Method

#### 4.2.1. Construct the Extension Judgment Matrix

#### 4.2.2. Calculation of Comprehensive Extension Judgment Matrix and the Weight Vector

- (1)
- The normalized eigenvectors with positive components corresponding to the maximum eigenvalues of ${A}^{-}$ and ${A}^{+}$ are ${X}^{-}$ and ${X}^{+}$.
- (2)
- The values of k and m are calculated by ${A}^{-}={\left({a}_{ij}^{-}\right)}_{n\times n}$ and ${A}^{+}={\left({a}_{ij}^{+}\right)}_{n\times n}$:$$\{\begin{array}{c}k=\sqrt{{\displaystyle \sum _{j=1}^{n}\left(1/{\displaystyle \sum _{i=1}^{n}{a}_{ij}^{+}}\right)}}\\ m=\sqrt{{\displaystyle \sum _{j=1}^{n}\left(1/{\displaystyle \sum _{i=1}^{n}{a}_{ij}^{-}}\right)}}\end{array}$$
- (3)
- Judge the consistency of the matrix. If $0\le k\le 1\le m$, it shows that the consistency of the extension interval judgment matrix is better. However, when consistency is too low, measures should be taken to correct the judgement matrix or allow experts to re-judge until the requirements are met.
- (4)
- Finding the weight vector:$$S={\left({S}_{1},{S}_{2},\dots ,{S}_{nk}\right)}^{T}=<k{x}^{-},m{x}^{+}>$$

#### 4.2.3. Hierarchy Sorting

#### 4.3. Quantitative Analysis Method for Driving Track of UGV

#### 4.3.1. Phase Space Reconstruction Using the C-C Method

- (1)
- The standard deviation of the time series of the driving trajectory deviation is calculated by $\sigma $, and selecting $N$
- (2)
- The following three formulas are calculated:$$\overline{S}\left(t\right)=\frac{1}{16}{\displaystyle \sum _{m=2}^{5}{\displaystyle \sum _{j=1}^{4}S\left(m,{r}_{j},t\right)}}$$$$\mathsf{\Delta}\overline{S}\left(t\right)=\frac{1}{4}{\displaystyle \sum _{m=2}^{5}\mathsf{\Delta}S(m,t)}$$$${S}_{cor}\left(t\right)=\mathsf{\Delta}\overline{S}\left(t\right)+\left|\mathsf{\Delta}\overline{S}\left(t\right)\right|$$The time variable t takes the natural number less than 200, and $S\left(m,{r}_{j},t\right)$, $\mathsf{\Delta}S\left(m,t\right)$ as follows:$$S\left(m,{r}_{j},t\right)=\frac{1}{t}{\displaystyle \sum _{s=1}^{t}\left[{C}_{s}\left(m,{r}_{j},t\right)-{C}_{s}^{m}(1,{r}_{j},t)\right]}\text{\hspace{1em}}m=2,3,4,5$$$$\mathsf{\Delta}S\left(m,t\right)=\mathrm{max}\left\{S\left(m,{r}_{j},t\right)\right\}-\mathrm{min}\left\{S\left(m,{r}_{j},t\right)\right\}$$
- (3)
- Drawing according to the results of the calculation:
- (1)
- The first minimum value of $\mathsf{\Delta}S\left(m,t\right)$ is t and corresponding to the best time delay.
- (2)
- The first zero point t of $\overline{S}(t)$ is the best time delay t.
- (3)
- The minimum value t of the ${S}_{cor}(t)$ corresponding to the time window ${\tau}_{\varpi}$. The greater the trajectory deviation change $D\left(t\right)$ from the actual trajectory corresponds to the ideal trajectory that made the phase space fuzzier and the dimensions larger.

#### 4.3.2. Calculation of the Lyapunov Index

- (1)
- The fast Fourier transform of the time series $\left\{x\left({t}_{i}\right),i=1,\text{}2,\text{}\cdots ,N\right\}$ of the driving trajectory deviation of the UGV is calculated using the fast Fourier transform (FFT), and the average period of $P$ is calculated.
- (2)
- The time delay $\tau $ and the embedding dimension $m$ are calculated by the C-C method.
- (3)
- Reconstructing phase space $\{{Y}_{j},j=1,2,\dots ,M\}$ according to time delay $\tau $ and embedded dimension $m$.
- (4)
- Finding the nearest neighbor point ${Y}_{\widehat{j}}$ of each point ${Y}_{j}$ in the phase space and limit the transient separation, that is:$${d}_{j}\left(0\right)=\mathrm{min}\Vert {Y}_{j}-{Y}_{\widehat{j}}\Vert \text{\hspace{1em}}\left|j-\widehat{j}\right|>P$$
- (5)
- For each point in the phase space ${Y}_{j}$, the distance after the $i$ discrete time steps corresponding to the adjacent point is calculated:$${d}_{j}\left(i\right)=\left|{Y}_{j+i}-{Y}_{\widehat{j}+i}\right|\text{\hspace{1em}}i=1,\text{}2,\text{}\cdots \text{},\mathrm{min}\left(M-j,M-\widehat{j}\right)$$
- (6)
- For each $i$, we calculate the average $y\left(i\right)$ of all $j$, that is:$$y\left(i\right)=\frac{1}{q\mathsf{\Delta}t}{\displaystyle \sum _{j=1}^{q}In{d}_{j}\left(i\right)}$$

#### 4.3.3. Design Ideal Trajectory

#### 4.4. The Grey Relational Analysis Method

#### 4.4.1. The Index Quantitative Standard

#### 4.4.2. Establishing Reference Series and Comparison Series

#### 4.4.3. Calculation of the Correlation Coefficient of Reference Series

#### 4.4.4. Calculate Correlation Degree of Comprehensive Evaluation

## 5. Experimental Verification

#### 5.1. Data Acquisition of the Driving Track of an UGV

#### 5.2. Quantitative Analysis

#### 5.3. Comprehensive Evaluation of Intelligent Behavior

#### 5.3.1. Determine the Weight Coefficient of Each Index

#### 5.3.2. Comprehensive Evaluation of Grey Relational Analysis Method

## 6. Conclusions

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Xi, L.; Zhang, X.; Sun, C.; Wang, Z.; Hou, X.; Zhang, J. Intelligent Energy Management Control for Extended Range Electric Vehicles Based on Dynamic Programming and Neural Network. Energies
**2017**, 10, 1871. [Google Scholar] [CrossRef] - Vidhi, R.; Shrivastava, P.; Sciubba, E. A Review of Electric Vehicle Lifecycle Emissions and Policy Recommendations to Increase EV Penetration in India. Energies
**2018**, 11, 483. [Google Scholar] [CrossRef] - Urmson, C.; Baker, C.; Dolan, J.; Rybski, P.; Salesky, B.; Whittaker, W.L.; Ferguson, D.; Darms, M. Autonomous driving in traffic: Boss and the urban challenge. AI Mag.
**2009**, 30, 17–29. [Google Scholar] [CrossRef] - Krotkov, E.; Fish, S.; Jackel, L.; McBride, B.; Perschbacher, M.; Pippine, J. The DARPA Percept OR evaluation experiments. Auton. Robot.
**2007**, 22, 19–35. [Google Scholar] [CrossRef] - Behringer, R.; Sundareswaran, S.; Gregory, B.; Elsley, R.; Addison, B.; Guthmiller, W.; Daily, R.; Bevly, D. The DARPA grand challenge development of an autonomous vehicle. In Proceedings of the 2004 IEEE Intelligent Vehicles Symposium, Parma, Italy, 14–17 June 2004; pp. 226–231. [Google Scholar]
- Sun, Y.; Xiong, G.; Song, W.; Gong, J.; Chen, H. Test and evaluation of autonomous ground vehicles. Adv. Mech. Eng.
**2014**, 2014. [Google Scholar] [CrossRef] - Miller, I.; Lupashin, S.; Zych, N.; Moran, P.; Schimpf, B.; Nathan, A.; Garcia, E. Cornell University’s 2005 DARPA grand challenge entry. J. Field Robot.
**2006**, 23, 625–652. [Google Scholar] [CrossRef] - Junqing, W.; Dolan, J.M. A robust autonomous freeway driving algorithm. In Proceedings of the 2009 IEEE Intelligent Vehicles Symposium, Xi’an, Shaanxi, China, 3–5 June 2009; pp. 1015–1020. [Google Scholar]
- Rui, L. Research on Path Tracking Algorithm and Lower Control System for Intelligent Vehicle; Beijing University of Technology: Beijing, China, 2013. [Google Scholar]
- Wildermuth, D.; Wolf, H. Professional ground robotic competitions from an educational perspective: A consideration using the example of the European land robot trial (ELROB). In Proceedings of the 2012 6th IEEE International Conference Intelligent Systems, Sofia, Bulgaria, 6–8 September 2012; pp. 399–405. [Google Scholar]
- Geiger, A.; Lauer, M.; Moosmann, F.; Ranft, B.; Rapp, H.; Stiller, C.; Ziegler, J. Team Annie WAY’s entry to the Grand Cooperative Driving Challenge 2011. IEEE Trans. Intell. Transp. Syst.
**2012**, 13, 1008–1017. [Google Scholar] [CrossRef] - Lauer, M.; Gerrits, A. Next steps for the grand cooperative driving challenge: ITS Events. IEEE Intell. Transp. Syst. Mag.
**2009**, 1, 24–32. [Google Scholar] [CrossRef] - Yang, S.; He, Y. Grading unmanned ground vehicles in terms of intelligence. Sci. Technol. Rev.
**2017**, 35, 80–83. [Google Scholar] - Sun, Y.; Xiong, G.M.; Chen, H.Y.; Wu, S.B.; Gong, J.W.; Jiang, Y. A cost function-oriented quantitative evaluation method for unmanned ground vehicles. Adv. Mater. Res.
**2011**, 301–303, 701–706. [Google Scholar] [CrossRef] - Sun, Y.; Tao, G.; Xiong, G.; Chen, H. The FUZZY-AHP evaluation method for unmanned ground vehicles. Appl. Math. Inf. Sci.
**2013**, 7, 653–658. [Google Scholar] [CrossRef] - Xiong, G.; Zhao, X.; Liu, H.; Wu, S.; Gong, J.; Zhang, H.; Tan, H.; Chen, H. Research on the Quantitative Evaluation System for Unmanned Ground Vehicles. In Proceedings of the 2010 IEEE Intelligent Vehicles Symposium, University of California, San Diego, CA, USA, 21–24 June 2010; pp. 523–527. [Google Scholar]
- Feng, Y.X.; Gao, Y.C.; Song, X.; Tan, J. Equilibrium Design Based on Design Thinking Solving: An Integrated Multicriteria Decision-Making Methodology. Adv. Mech. Eng.
**2013**, 5. [Google Scholar] [CrossRef] - Vaidya, O.S.; Kumar, S. Analytic hierarchy process: An overview of applications. Eur. J. Oper. Res.
**2006**, 169, 1–29. [Google Scholar] [CrossRef] - Chen, H.Y.; Xiong, G.M.; Gong, J.W.; Jiang, Y. Introduction to Self-Driving Car; Beijing Institute of Technology Press: Beijing, China, 2014; pp. 32–64. [Google Scholar]
- Chen, Y.; Mei, Y.; Yu, J.; Su, X.; Xu, N. Three-dimensional unmanned aerial vehicle path planning using modified wolf pack search algorithm. Neurocomputing
**2017**, 266, 445–457. [Google Scholar] - Liu, Y.; Xu, W.; Dobaie, A.M.; Yan, Z. Autonomous road detection and modeling for UGVs using vision-laser data fusion. Neurocomputing
**2018**, 275, 2752–2761. [Google Scholar] [CrossRef] - Saaty, T.I. The Analytic Hierarchy Process; McGraw Hill Inc.: New York, NY, USA, 1980. [Google Scholar]
- Lv, J.H.; Lu, J.A.; Chen, S.H. Analysis and Application of Chaotic Time Series; Wuhan University Press: Wuhan, China, 2002. [Google Scholar]
- Li, K.Q.; Wang, Y.J.; Gao, F. Driving assistant system based on ITS. J. Jiangsu Univ.
**2005**, 26, 294–297. [Google Scholar] - You, F. Study on Autonomous Lane Changing and Autonomous Overtaking Control Method of Intelligent Vehicle. Ph.D. Thesis, Jilin University, Changchun, China, 2005. [Google Scholar]

**Table 1.**Design of unmanned ground vehicle (UGV) test content scene (global positioning system (GPS)).

Test content | Test Layer | Simple Scene | Complex Scene |

Environmental perception | Traffic sign recognition | Understanding of traffic signs | |

Lane line recognition | Object tracking | ||

Object recognition | Target distance and velocity perception | ||

Whistle perception | Map drawing | ||

Decision control | Straight | Overtaking, switching | |

Turn | Static dynamic obstacle avoidance | ||

Accelerate | Emergency obstacle avoidance | ||

Stop | Complex lane keeping | ||

Intelligent interaction | Passenger interaction | Pedestrian interaction | |

Location sharing | Environmental interaction | ||

Information sharing | Traffic equipment interaction | ||

Single car interaction | Multi vehicle interaction | ||

Navigation | Simple path planning | Complex path planning | |

Optimal path | Special weather navigation | ||

Complete only by GPS | GPS signal loss | ||

Straight navigation | Road, off-road navigation |

Evaluation Aspect | Smartness | Safety | Smoothness | Single Layer Weight |
---|---|---|---|---|

Smartness | <1, 1> | <1, 1.5> | <1.5, 2> | 0.52 |

Safety | <0.6667, 1> | <1, 1> | <1, 2> | 0.35 |

Smoothness | <0.5, 0.6667> | <0.5, 1> | <1, 1> | 0.13 |

Evaluation Aspect | Evaluation Factor | Quantitative Results V | ||||||
---|---|---|---|---|---|---|---|---|

k | u_{k} | a_{k} | j | u_{j} | $\mathit{\omega}$_{j} | Assignment Matrix | ||

Vehicle C_{1} | Vehicle C_{2} | Vehicle C_{3} | ||||||

1 | Smartness | 0.52 | 1 | Environmental perception | 0.47 | 8 | 6 | 5 |

2 | Path planning | 0.18 | 1.25 | 3.54 | 2.33 | |||

3 | Decision | 0.1 | 4.5 | 4 | 6 | |||

4 | Control | 0.1 | 2.15 | 1.73 | 2.82 | |||

5 | Interactive | 0.15 | 7 | 3.5 | 5.5 | |||

2 | Safety | 0.35 | 1 | Traffic safety | 0.55 | 2.33 | 2.45 | 1.55 |

2 | Information safety | 0.31 | 5 | 5 | 4 | |||

3 | Sensor safety | 0.08 | 5.5 | 6 | 3.5 | |||

4 | Redundancy | 0.06 | 4 | 3 | 3.5 | |||

3 | Smoothness | 0.13 | 1 | Smooth start | 0.2 | 4 | 4.5 | 3.5 |

2 | Smooth speed | 0.46 | 6.5 | 5.5 | 6 | |||

3 | Smooth brake | 0.13 | 7 | 6.5 | 5.5 | |||

4 | Smooth trajectory | 0.21 | 1.32 | 4.53 | 2.66 |

Aspect | Element | a_{k} | Factor | $\mathit{\omega}$_{j} | Matrix | Results | ||||
---|---|---|---|---|---|---|---|---|---|---|

C_{1} | C_{1} | C_{2} | C_{1} | C_{2} | C_{3} | |||||

Environmental perception | Visual perception | 0.42 | Traffic sign recognition | 0.35 | 7.5 | 6 | 6.5 | 0.82 | 0.58 | 0.52 |

Traffic signal lamp recognition | 0.28 | 7 | 8 | 7.5 | ||||||

Lane line recognition | 0.22 | 7 | 6.5 | 8 | ||||||

Object recognition | 0.15 | 8.5 | 7.5 | 7.5 | ||||||

Radar perception | 0.38 | Target distance perception | 0.3 | 8 | 7 | 6.5 | ||||

Target speed perception | 0.3 | 8.5 | 6.5 | 7.5 | ||||||

Obstacle perception | 0.25 | 7.5 | 8 | 6 | ||||||

3D reconstruction | 0.15 | 6 | 5.5 | 6 | ||||||

Sound perception | 0.2 | Traffic sound | 0.33 | 7 | 6.5 | 6 | ||||

Special sound | 0.36 | 8 | 7.5 | 7 | ||||||

Voice command | 0.31 | 7 | 7.5 | 7 |

Vehicle | Smartness | Safety | Smoothness | Total Score | Rank |
---|---|---|---|---|---|

Vehicle 1 | 93.35 | 66.68 | 96.20 | 84.39 | 1 |

Vehicle 2 | 56.32 | 62.17 | 74.25 | 60.70 | 3 |

Vehicle 3 | 57.51 | 78.22 | 74.92 | 67.02 | 2 |

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Sun, Y.; Yang, H.; Meng, F.
Research on an Intelligent Behavior Evaluation System for Unmanned Ground Vehicles. *Energies* **2018**, *11*, 1764.
https://doi.org/10.3390/en11071764

**AMA Style**

Sun Y, Yang H, Meng F.
Research on an Intelligent Behavior Evaluation System for Unmanned Ground Vehicles. *Energies*. 2018; 11(7):1764.
https://doi.org/10.3390/en11071764

**Chicago/Turabian Style**

Sun, Yang, He Yang, and Fei Meng.
2018. "Research on an Intelligent Behavior Evaluation System for Unmanned Ground Vehicles" *Energies* 11, no. 7: 1764.
https://doi.org/10.3390/en11071764