A Synthesis of Algorithms Determining a Safe Trajectory in a Group of Autonomous Vehicles Using a Sequential Game and Neural Network

Abstract

This paper presents a solution to the problem of providing an autonomous vehicle with a safe control task when moving around many other autonomous vehicles. This is achieved by developing an appropriate computer control algorithm that takes into account the possible risk of a collision resulting from both the impact of environmental disturbances and the imperfection of the rules of maneuvering in situations where many vehicles pass each other, giving the control process a decisive character. For this purpose, three types of algorithms were synthesized: kinematic and dynamic optimization with neural domains, as well as sequential game control of an autonomous vehicle. The control algorithms determine a safe trajectory, which is implemented by the actuators of the autonomous vehicle. Computer simulations of the control algorithms in the Matlab/Simulink software allow for their comparative analysis in terms of meeting the criteria for the optimality and safety of an autonomous vehicle when passing a larger number of other autonomous vehicles. For this purpose, scenarios of multidirectional and one-way traffic of autonomous vehicles were used.

1. Introduction

Intelligent autonomous vehicles have increased their capabilities for highly automated driving in complex environments. Currently, the research is focused on the implementation of strategies that improve safety and comfort and optimize energy consumption. In addition, it is important for autonomous vehicles to be able to navigate dynamic urban environments with the ability to avoid obstacles involving vulnerable road users, such as pedestrians and cyclists, and to perform joint maneuvers between automated and semi-automated vehicles.

1.1. The State of Knowledge of Autonomous Vehicles Route Planning

A review of the literature on the subject of autonomous vehicle traffic control was carried out chronologically for the last fifteen years.

Dolgov et al. [1] presented an algorithm for planning the path of an autonomous vehicle operating in an unknown environment, in which obstacles are detected online by vehicle sensors; a method for numerical non-linear optimization was used to achieve its optimum implementation.

An urban traffic planning algorithm based on a fast-exploring random tree was developed by Kuwata et al. [2]. Additionally, Luo et al. presented [3] an algorithm for adaptive cruise control using the predictive control model. On the other hand, the method of controlling a line platoon as a system of single vehicles moving together, assuming a train configuration without material coupling, was presented by Contet et al. [4].

In their paper, Khodayari et al. [5] proposed the use of a model predictive control controller to predict the future behavior of the leading vehicle and then control the acceleration of the following vehicle.

The use of a two-person non-zero-sum non-cooperative game under complete and incomplete information in the lane change algorithm was presented by [6] Talebpour et al.

Gonzalez et al. [7] conducted a valuable review of intelligent vehicle traffic planning techniques, and Bast et al. offered [8] a review of practical algorithms for determining vehicle routes in metropolitan transport networks using the graph technique.

A control algorithm for following a car in a platoon on a straight highway as a multi-agent dynamic system subject to traffic safety constraints was described in [9] by Gong et al.

Marin-Plaza et al. presented [10] a method of path planning based on time elastic bands in a real research platform using the Ackermann model and rotating the inner wheel slightly more sharply than the outer wheel to reduce tire slippage. The use of a distributed adaptive synchronization strategy for vehicle cooperation, whose stability was assessed using the Lyapunov–Krasovskii method, was presented in [11] by Petrillo et al. However, Huang Z. et al. [12] proposed an algorithm for cooperative vehicle driving that combines the artificial potential field method with predictive control and which synchronously performs both path planning and traffic control. In another paper [13], Huang Y. et al. designed the planning and tracking of vehicle movement using the resistance of an artificial potential field, and the current comparison method was used to find a collision-free path. In addition, Li and Vorobeychik [14] proposed a vehicle pathing plan based on game theory elements, using the Nash equilibrium point and a multi-agent path-planning formula in vehicle cooperation mode. Another solution for determining the vehicle trajectory was presented by Chen et al. [15] in the form of a polynomial planning algorithm guided by a histogram of a vector field. Elsewhere, Said et al. [16] proposed a trajectory planning algorithm that calculates possible paths according to a set of predefined navigation map points.

As a tool for supporting vehicle path determination, Braud et al. [17] developed a maneuver design language to structurally describe maneuvers in a machine-readable format. In contrast, Dinh et al. [18] presented a trajectory controller that evaluates the planned trajectory in terms of various safety rules.

A list of different methods for planning the path of a mobile robot was produced by Xu et al. in [19] in the form of algorithms based on sampling, nodes, and mathematical models inspired by biology in the form of evolutionary and neural networks and multi-fusion-based algorithms.

More recently, Abdallaoui et al. [20] proposed a route-planning algorithm for automated vehicles, specifically for emergency situations, based on the decision-support framework. Moreover, Bautista-Camino et al. [21] presented a local path planner for an autonomous vehicle based on dynamic attraction, mapping the motion behavior of living beings and taking into account four levels of acceleration. The project of a predictive controller for the safe driving of an autonomous vehicle in uncertain conditions, which takes into account other road users, such as pedestrians and cyclists, whose intentions cannot be precisely known, was presented by Batkovic [22]. Kim et al. [23] described two-stage trajectory planning. The long-term stage long-term planner generates a safe, comfortable, and dynamically feasible trajectory. In the short-term stage, the trajectory is optimized using particle swarm methods.

Summing up the above literature in the field of route-planning methods for autonomous vehicles, it can be stated that these studies aimed to determine the optimal trajectory of an autonomous vehicle with various optimization criteria. However, the main criterion, apart from safety, was to allow the vehicle to reach its destination in the shortest possible time. Therefore, in order to compare the new algorithms presented by the author in this paper with those from the previous literature, this paper develops a representative-comparative-universal KO kinematic optimization algorithm.

1.2. The Thesis and Objectives of This Paper

In the literature, there are no works dealing with the synthesis of advanced control methods for a larger group of objects. Therefore, the thesis of the paper is to demonstrate that it is possible to synthesise algorithms for a safe trajectory of an autonomous vehicle moving in a larger group of vehicles, which will take into account the control constraints in the decision-making process, resulting from the ambiguity of maneuvering rules and environmental disturbances, using sequential game methods and artificial neural network. In the synthesis of algorithms for determining the safe trajectory of an autonomous vehicle, the hierarchical structure of the entire control system in the group of autonomous vehicles, which is shown in Figure 1, should be taken into account.

The scientific objective is the synthesis of new algorithms for determining a safe trajectory in a group of autonomous vehicles using sequential games and dynamic optimization.

The research objective consists of an experimental comparative analysis of computer simulation studies of traffic scenarios at various set safe distances.

1.3. Paper Contents

The paper content is presented as follows. In Section 1, we introduce the development of the design and operation of autonomous vehicles in recent years and present the research objective. Section 2 shows the synthesis of the safe-trajectory-determining algorithms. Section 3 describes the simulation used to train and validate three designed control algorithms in a group of autonomous vehicles. After that, the experimental analyses of the autonomous vehicle safe control trajectory algorithms are given in Section 4. Section 5 concludes this paper and looks at future directions for improvement.

2. Safe Trajectory Determining Algorithms

The synthesis of trajectory determination algorithms in a group of autonomous vehicles was carried out on the basis of an appropriate, most realistic model of the control process. Since the process of controlling an autonomous vehicle includes both elements of the kinematics and dynamics of movement and also elements of cooperation with other autonomous vehicles, the most general model of such a process is the multiobject differential game model.

The author used his research experience in the synthesis of controlling autonomous robots moving and marine objects [24,25].

In general, with regard to trajectory planning, the following types of trajectories can be distinguished in the group of autonomous vehicles:

• Kinematic trajectory, taking into account the dynamics of the vehicle in the form of advance time of the course or speed change manoeuvre;
• Dynamic trajectory, taking into account vehicle dynamics in the form of first-order differential equations of process state;
• Dynamic trajectory, taking into account possible changes in the course and speed of another autonomous vehicle.

Table 1. General characteristics of trajectory determination algorithms in a group of autonomous vehicles.

Control Process Model	Control Theory Method	Algorithm
Kinematic	Linear Programming	Kinematic Optimization KO
Dynamic	Dynamic Programming Neural Network	Dynamic Optimization with Neural Network DONN
Game	Triple Linear Programming	Game Optimization GO

proposes three types of algorithms for determining the safe trajectory in a group of autonomous vehicles, referring to the kinematic, dynamic, and game control process models, respectively.

Figure 2 shows the variables describing the kinematic motion of an autonomous vehicles’ group.

The next three subsections present, in a synthetic form, a description of three trajectory determination algorithms in a group of autonomous vehicles.

2.1. Kinematic Optimization KO Algorithm

The kinematic model takes into account the kinematic equations of autonomous vehicles in a multistage approach, corresponding to the sequence of subsequent maneuvers. The dynamics of autonomous vehicles are presented in the form of maneuver advance time t_a, which consists of the course change time, which is equal to the setting time t_s of the course to the new set value. According to Nise [26] the setting time of an inertial object with a time constant T, as the time to reach the setpoint, is equal to the four time constants of the object t_s = 4T.

At a single stage of movement, the optimal collision avoidance maneuver to maintain the previously established safe passing distance δ_s by optimally changing the course on the right χ^*_0,r or on the left χ^*_0,l or optimally reducing the speed σ₀^*, is determined from the area of acceptable maneuvers (green), the structure of which is shown in Figure 3.

A circle with a radius equal to the safe approach distance δ_s is drawn around the position of each vehicle. Tangents to these circles, drawn from our autonomous vehicle, define in a semicircle with a radius equal to the speed of our autonomous vehicle; they define areas of dangerous approach at a distance less than δ_s (red color) and the safe passing area at distances greater than δ_s (green color).

The distance between vehicles is measured by a radar device or other device.

The Simplex linear programming method was used to calculate the optimal course and speed values in a limited area of acceptable maneuvers.

The criterion for the quality of optimal vehicle control is reaching the nearest turning point with the least loss of distance for anti-collision maneuvering. These losses are the smallest at the maximum vehicle speed component in the direction leading to this point. When the vehicle is at a constant speed, minimizing the road losses s* is equivalent to minimizing the time t* to reach the turning point:

$$\underset{\max {\sigma }_{0\mathrm{X}}}{\min } \mathrm{s} \left[\left({\mathrm{X}}_{\mathrm{n}}(\mathrm{t}),{\mathrm{Y}}_{\mathrm{n}}(\mathrm{t})\right), \left({\mathrm{X}}_{\mathrm{k}},{\mathrm{Y}}_{\mathrm{k}}\right)\right] =\mathrm{s}*\to \mathrm{t}*, {\Delta }_{\mathrm{K}\mathrm{O}}$$

(1)

The effect of the minimal-time and anti-collision maneuvering of the vehicle is the deviation Δ_KO of this safe trajectory from the set trajectory, which is read from the calculated optimal trajectory.

The determination of a safe kinematic trajectory as a multistage decision-making process is presented in the block diagram of the KO program in Figure 4.

2.2. Dynamic Optimization with a Neural Network DONN Algorithm

The dynamic model contains equations of the state of the control process, describing the dynamic properties of the autonomous vehicle and constraints in the form of motion domains assigned to each encountered autonomous vehicle. The circle and hexagonal domains are formed by an early trained three-layer neural network with hyperbolic tangent and logistic activation functions.

MATLAB Neural Network Toolbox software was used to design the NN network, and an error propagation algorithm with adaptive learning rate and momentum was used to train it. The training data were prepared by simulating several scenarios and recording the corresponding expected network responses from approximately 500 experienced drivers. For each scenario, the driver chose the best option at his discretion; that is, subjectively, in accordance with good practice, the driver chose an anti-collision maneuver to change course and/or car speed. The learned network thus represents the average experience of a larger population of drivers.

With regard to the multi-agent reinforcement learning described in the literature, the author of the paper used ordinary supervised learning in the synthesis of this algorithm.

The vehicle control quality criterion is the smallest path deviation s from the nearest route turning point, which is achieved with the shortest time of reaching this point:

$$\underset{\mathrm{t}}{\min } \mathrm{s} \left[\left({\mathrm{X}}_{\mathrm{n}}(\mathrm{t}),{\mathrm{Y}}_{\mathrm{n}}(\mathrm{t})\right), \left({\mathrm{X}}_{\mathrm{k}},{\mathrm{Y}}_{\mathrm{k}}\right)\right] = \mathrm{s}*\to \mathrm{t}*,\hspace{1em}{\Delta }_{\mathrm{D}\mathrm{O}}$$

(2)

The determination of a safe dynamic trajectory as a dynamic programming process is presented in the block diagram of the DONN program in Figure 5.

The procedure 1 computer program representing the Hexagon Domain Calculation in the MATLAB/Simulink software used to calculate the dynamic trajectory is presented in Figure 6.

2.3. Game Optimization GO Algorithm

The game model of the process is a reduction in the general model from the differential game of many participants to the multistep positional game of the autonomous vehicles’ group. First, the area of acceptable maneuvers for our autonomous vehicle was determined in relation to the other n autonomous vehicles using the method shown in Figure 3. Then, using the same method, the sets of allowable maneuvers for each of the n passing autonomous vehicles were determined in relation to our autonomous vehicle.

The dynamics of the autonomous vehicles are presented in the form of maneuver advance time, which consists of the course change time, approximately equal to three time constants of the autonomous vehicle as the object of automatic control.

The criterion for the quality of vehicle control is the smallest deviation of the track s from the given nearest navigation point, which is achieved for appropriate changes in the course and speed of autonomous vehicles in the absence of cooperation between the vehicles:

$$\underset{{\sigma }_0,{\chi }_0}{\min } \underset{{\sigma }_n,{\chi }_n}{\max } \underset{{\sigma }_0^{\mathrm{n}},{\chi }_0^{\mathrm{n}}}{\min } \mathrm{s} \left[\left({\mathrm{X}}_{\mathrm{n}}(\mathrm{t}),{\mathrm{Y}}_{\mathrm{n}}(\mathrm{t})\right), \left({\mathrm{X}}_{\mathrm{k}},{\mathrm{Y}}_{\mathrm{k}}\right)\right] =\mathrm{s}*\to \mathrm{t}*, {\Delta }_{\mathrm{G}\mathrm{O}\mathrm{n}\mathrm{c}}$$

(3)

under the conditions of the cooperative maneuvering in a group of autonomous vehicles:

$$\underset{{\sigma }_0,{\chi }_0}{\min } \underset{{\sigma }_{\mathrm{n}},{\chi }_{\mathrm{n}}}{\min } \underset{{\sigma }_0^{\mathrm{n}},{\chi }_0^{\mathrm{n}}}{\min } \mathrm{s} \left[\left({\mathrm{X}}_{\mathrm{n}}(\mathrm{t}),{\mathrm{Y}}_{\mathrm{n}}(\mathrm{t})\right), \left({\mathrm{X}}_{\mathrm{k}},{\mathrm{Y}}_{\mathrm{k}}\right)\right] =\mathrm{s}*\to \mathrm{t}*, {\Delta }_{\mathrm{G}\mathrm{O}\mathrm{c}}$$

(4)

The determination of a safe game trajectory as a multistep game decision-making process is presented in the block diagram of the GO program in Figure 7.

3. Computer Simulation

The developed KO, DO, and GO algorithms for trajectory determination in a group of autonomous vehicles were tested in the MATAB/Simulink software in simulation studies of two traffic scenarios: multidirectional traffic and one-way traffic with seven other autonomous vehicles.

The computer simulation of the vehicle traffic was carried out for three values of δ_s of the safe distance between autonomous vehicles.

3.1. Multidirectional Traffic Scenario

The multidirectional traffic scenario of the autonomous vehicles is described in

Table 2. Data of the autonomous vehicles in multidirectional traffic.

Autonomous Vehicle n	Speed σn (m/s)	Course χn (°)	Position Component Xn (m)	Position Component Yn (m)
0	20	0	0	0
1	14	90	−520	760
2	18	240	410	1360
3	16	260	310	760
4	25	0	650	60
5	8	90	−640	710
6	24	280	910	1060
7	10	170	−210	1500

, with the data in the form of the speed σ_n, the course χ_n, and the position components (X_n,Y_n).

3.1.1. Kinematic Optimization Trajectory

The simulation results of the KO algorithm in the multidirectional traffic scenario are presented in Figure 8 in the form of our autonomous vehicle safe trajectories in order to maintain a safe passing distance of another autonomous vehicle of 20 m, 50 m and 100 m. The evaluation of the trajectory is its final deviation Δ from the course leading to the nearest turning point of the route.

3.1.2. Dynamic Optimization Trajectory

The simulation results of the DO algorithm in the multidirectional traffic scenario are presented in Figure 9 in the form of our autonomous vehicle safe trajectories in order to maintain a safe passing distance of another autonomous vehicle of 20 m, 50 m and 100 m. The evaluation of the trajectory is its final deviation Δ from the course leading to the nearest turning point of the route.

3.1.3. Game Optimization Trajectory

The simulation results of the GO_nc and GO_c algorithms in the multidirectional traffic scenario are presented in Figure 10 and Figure 11 in the form of our autonomous vehicle safe trajectories in order to maintain a safe passing distance of another autonomous vehicle of 20 m, 50 m and 100 m. The evaluation of the trajectory is its final deviation Δ from the course leading to the nearest turning point of the route.

3.2. One-Way Traffic Scenario

The one-way traffic scenario of autonomous vehicles is described in

Table 3. Data of autonomous vehicles in one-way traffic.

Autonomous Vehicle n	Speed σn (m/s)	Course χn (°)	Position Component Xn (m)	Position Component Yn (m)
0	0	0	10.1	6.9
1	10	0	−9.9	7.6
2	5	0	10.0	10.0
3	10	0	9.8	10.0
4	10	0	0	10.0
5	15	0	10.2	28.2
6	15	0	−10.2	28.1
7	10	0	0	36.1

by their data in the form of the speed σ_n, the course χ_n, and the position components (X_n,Y_n).

3.2.1. Kinematic Optimization Trajectory

The simulation results of the KO algorithm in the one-way traffic scenario are presented in Figure 12 in the form of our autonomous vehicle safe trajectories in order to maintain a safe passing distance of another autonomous vehicle of 2.5 m, 5 m, and 10 m. The evaluation of the trajectory is its final deviation Δ from the course leading to the nearest turning point of the route.

3.2.2. Dynamic Optimization Trajectory

The simulation results of the DO algorithm in the one-way traffic scenario are presented in Figure 13 in the form of our autonomous vehicle safe trajectories in order to maintain a safe passing distance of another autonomous vehicle of 2.5 m, 5 m, and 10 m. The evaluation of the trajectory is its final deviation Δ from the course leading to the nearest turning point of the route.

3.2.3. Game Optimization Trajectory

The simulation results of the GO_nc and GO_c algorithms in the one-way traffic scenario are presented in Figure 14 and Figure 15 in the form of our autonomous vehicle safe trajectories in order to maintain a safe passing distance of another autonomous vehicle of 2.5 m, 5 m, and 10 m. The evaluation of the trajectory is its final deviation Δ from the course leading to the nearest turning point of the route.

4. Discussion

The best way to discuss the properties of the tested algorithms is to compare the resulting safe trajectories of the autonomous vehicle, which are presented in Figure 16 and Figure 17.

4.1. Trajectory Comparison in Multidirectional Traffic

In the situation of the multidirectional traffic of autonomous vehicles, the safe trajectories determined according to the KO, DO, and GO algorithms did not differ much for small values of the previously determined safe distances δ_s in real traffic conditions.

The value of the safe distance of vehicle passing was adopted as a measure of traffic conditions. When the traffic conditions deteriorated, for example, limited visibility during fog, then the value of the required safe passing distance ds necessarily increased, and the non-cooperative algorithm, GOnc, taking into account possible errors in steering other autonomous vehicles, ensured greater traffic safety at the expense of a greater deviation of the final trajectory of the traffic.

On the other hand, the GOc cooperative game algorithm was very effective in such situations.

The KO kinematic trajectory algorithm, despite the small final deviation, omitted the collision risk assessment caused by difficult traffic conditions.

A compromise solution was the DONN dynamic trajectory algorithm supported by a neural network evaluating the subjective characteristics of the decision maker (Figure 18).

4.2. Trajectory Comparison in One-Way Traffic

In one-way traffic, the algorithms behaved similarly; however, there were larger differences in the final deviation of the trajectory, both for small and large values of the safe passing distance δ_s. In deteriorating traffic conditions, the GOc cooperative game trajectory algorithm was the most effective (Fgure 18).

4.3. Comparison of the Final Trajectory Deviation

In the multi-directional traffic scenario, the deterioration of the driving conditions forcing the vehicle to increase the safe approach distance five times from 20 m to 100 m results in the largest 13-fold final trajectory deviation for the GO_nc algorithm and the smallest three-fold for the DONN algorithm. In the one-way traffic scenario, the deterioration of driving conditions forcing the four-fold increase in the safe approach distance between vehicles from 2.5 m to 10 m results in the largest eight-fold final trajectory deviation for the DONN algorithm and the smallest 1.5-fold for the GO_c algorithm.

For both scenarios, the most common compromise is to use the cooperative algorithm of the GO_c trajectory.

Comparing the obtained results in determining the dynamic trajectory of the DO and that of the quartering GO in relation to the kinematic trajectory of KO, which represent the existing methods of vehicle path optimization, the following conclusions can be reached:

• The final safe trajectory deviation is much greater for the DO and GO algorithms compared to the KO algorithm, but this comes at the expense of increasing the traffic safety of the autonomous vehicle to take into account the uncertainty of the environment and the non-cooperative behavior of other autonomous vehicles;
• In the case of the DO algorithm, the deviation increases 18 times;
• In the case of the GOc algorithm, the deviation increases only five times;
• In the case of the GOnc algorithm, the deviation increases as much as 26 times.

5. Conclusions

The analysis of the developed trajectory determination algorithms in a group of autonomous vehicles allowed the following final conclusions to be drawn:

• The synthesis of algorithms to determine safe trajectory in the group of autonomous vehicles using sequential games and dynamic optimization enabled the traffic condition to be taken into account;
• Using the variable domains of an autonomous vehicle generated by a previously trained neural network allowed the limited perception of the control algorithm to be taken into account.

The main result of this research is the preparation of several versions of control algorithms for practical implementation in the programmable logic controller PLC autonomous vehicle.

Future research should focus on the following tasks:

• Synthesis of an integrated dynamical-game control algorithm;
• Evaluation of control quality sensitivity to information inaccuracy and disturbances level.