Game Control Methods Comparison when Avoiding Collisions with Multiple Objects Using Radar Remote Sensing

: This article formulates the concept of games in the field of process control theory in marine sciences and reviews the literature on the possible applications of games. The possible types of game control processes for moving objects are presented. A computer-aided object safe control in the game environment, with an appropriate steering system, is described based on radar remote sensing in order to avoid collisions with many other objects that are encountered. First, the basic model of object movement in the game environment is presented as a differential game with many objects, described by appropriate game state equations, state and steering restrictions, and a quality control index in the form of an integral and final payment of the game. Next, the surrogate models of the differential game are described in detail for the development of practical computer control programs using positional and matrix game models. Particular attention, in each type of game, is paid to the aspect of cooperation or lack of cooperation between objects in making maneuvering decisions. A computer simulation illustrates these considerations with game control programs at a sea-crossing situation where multiple objects were encountered. Safe object trajectories are compared using two methods of game control using positional and matrix game models while also considering cases with cooperation or non-cooperation of objects.


Introduction
One of the most important problems that occurs in maritime transport is the prevention of collisions with objects, as described by Bist and Cahill [1,2]. To this end, it is mandatory that all objects are equipped with the remote sensing Automatic Radar Plotting Aid (ARPA) system, which tracks the movement of the objects that are encountered, calculates the parameters of their approach and allows for a trial maneuver for the most dangerous object to take into account the rules from the International Regulations for Preventing Collisions at Sea (COLREGs). Then, whether such a course or speed change maneuver is safe for other objects is assessed. If it is not secure, then the selection and trial maneuver are repeated for a new dangerous object. This solution to the anti-collision problem only works in situations where several objects pass each other. When a larger number of objects pass, it is necessary to use computer-aided maneuvering decisions in order to calculate all the possible solutions of the task and to propose one of the best ones, i.e., the optimal solution. At the same time, the optimization is as good as the mathematical model of the object control process is adequate. There are deterministic models described in References [3][4][5][6][7][8][9][10][11][12][13][14]; Blaich et al. [15] introduced a probabilistic model and Lazarowska and Tomera [16,17] presented a heuristic, notably, the swarm of particles model, as well as models using artificial intelligence, which were used in References [18][19][20][21][22][23][24][25][26] and game theory formulated, which was used in References [27][28][29][30][31].
The purpose of this review is to refer the considerations to the last method mentioned above through a detailed comparative analysis of the methods of game object control in collision situations with many objects passing by using radar remote sensing. The main premise for these considerations is that the navigator's subjectivity in assessing the navigational situation, taking into account the rules of the sea law COLREGs, as well as the possibility of making a mistake and contributing to a collision situation as a conflict situation, has a great impact here. Game theory, which is a chapter of modern mathematics, including the theory of accidental events and the construction and analysis of their models, comes to the rescue. The conflict may be military, political, social, or economic, where there is a game with nature and there is the application of object steering, that is, when they interact, the impact disturbances or other objects can be seen. Therefore, it is appropriate to treat the process of passing objects safely as a game control, taking into account the cooperation or lack of cooperation between objects. The problem of collision-free sea control strategies has already appeared in Isaacs [32], the creator of differential game theory, and has been developed by many authors, especially object control in conditions of uncertainty. According to the authors [33][34][35], determining anticollision strategies is easy, except for the uncertainty of the data, the results from harsh weather, high sea levels, as well as the actions of other objects and the navigator's subjectivity in maneuvering decisions and imprecise character recommendations of international collision regulations COLREGs. Dockner et al. and Baba et al. [36,37] stated that the topic of defining safe strategies is still valid, due to the constantly increasing vessel traffic in individual water bodies; on the one hand, increasing the requirements of safe navigation and the safety of the environment; on the other, increasing the operational capacity of computer-assisted navigators. According to Breton et al. and Sanchez-Soriano [38,39], the largest type of game relates to the motion control of objects, including autonomous and non-autonomous marine objects, which are differential games.
This review concerns the comparative analysis of the applications of certain types of games, leading to the synthesis of algorithms of computer-aided navigator decisions in the safe maneuvering of an object in difficult and complex collision situations at sea.

General Description
On the basis Haurie et al. and Mesterton-Gibbons [40][41][42], the movement of objects in time describes the system of state equations: (1) where x i (t) is the function that describes the state of the control process; u j (t) is the function that describes how the j-th object controls in the form of its i-th strategies; t is the time; N is the permissible object strategies number; J is the object's number.
According Osborne and Broek et al. [43,44], for the class often considered in the control technique of non-coalition games, the most beneficial conduct of the player is to minimize the goal function in the form of a functional: (2) where f oj is the integral payoff and f is the final payoff.
The strategy of the j-th object leads to a game balance situation that is determined by Nash [45] equilibrium points. The optimal control of the j-th player, as an object or transport process, is determined from the following relationship: Based on [46][47][48], it can be assumed that the task of developing the game control system is solved in two stages, corresponding to sufficient and necessary conditions to resolve the game J participants. Generally, there are two types of game controls: programming u j (t) and positioning u j [x(t)]. The main game control systems are positioning systems to control the position of objects, and also the feedback systems that represent the positional game.

Types of Object Traffic Game Control
The movement of objects takes place under the influence of controls on their movement, from U sets of possible strategies: -possible strategies set of own object (OO); -possible strategies set of j-th object.
• -leads to the shortest distance between subjects, often during the dangerous objects approach or when transferring cargo at sea.
• -this means an anti-collision maneuver to achieve the value of the smallest distance to j-th object, which is greater than the apparent safety distance D s that is determined under the given conditions: where D j min is the distance of the closest point of approach; D j is the distance to the encountered j-th object; D s is the safe distance of approach.
• -this means stabilizing the course or trajectory.
The following types of object traffic controls can be distinguished: This happens in cases of incorrect assessment of the proximity situation by one object, while the other object does not observe the situation. It can also take place when one of the objects has a malfunctioning ARPA system or when the weather conditions are very bad, leading to the disappearance of the object's echoes. Case 3: Avoiding collisions by the following:

•
Maneuvers of own object , for example, by changing a course Δψ and/or speed ΔV, • Maneuvers of the j-th object , • Cooperating maneuvers .

Case 4: Objects meeting .
Case 5: Stabilization of the course or trajectory .

Computer-Aided Object Safe Control in the Game Environment
Own object, moving course ψ at speed V, is equipped with an ARPA device, performs radar remote sensing of the encountered j-th object, moving course ψ j at speed V j , at distance D j and bearing N j (Figure 1).

Figure 1.
Displaying the situation of passing own object with multiple objects, in particular with the j-th object: X 0 , Y 0 -own object coordinates; X j , Y j -j-th object coordinates; X r , Y r -own object reference coordinates; p r -the reference position of own object; D j min -distance of the closest point of approach; T j min -time to the closest point of approach; and D s -a safe distance of approach.
The ARPA device only allows the movement of objects to be directed to the minimum necessary extent, which is the result of the COLREGs rules, and is limited to a trial maneuver in relation to a small number of objects, where the navigator's subjectivity in the assessment of steering decisions is highly important, often leading to collision situations. Therefore, it is advisable to supplement the ARPA system with the function of the computer-aided calculation of safe object passage because there are subsequent course and speed changes in relation to many objects that are passed ( Figure 2). The crucial part of the computer-aided navigator system, in the generation of control decisions, is the steering algorithm, which uses data from ARPA radar remote sensing and the appropriate process model, which calculates the safe trajectory of the object. In taking into account the game properties of the steering process, the most basic and adequate is the differential game model, which is most often used as a laboratory simulation model.
When designing practical real-time control algorithms, simpler surrogate models are used. Such surrogate models include the positional game model and matrix game model.

Basic Model of Differential Game
The COLREGs rules on transport safety mandate that all facilities must comply with and apply only to two objects, especially in good visibility at sea. In restricted visibility at sea, they are limited to very general recommendations that result from "good seagoing practice". In situations where multiple objects pass each other in bad weather conditions, the movement of objects often occurs with the close coordination of actions and leads to the risk of conflict and collision.
Therefore, in analyzing the work by Bressan et al. and Gromova et al. [49,50], it is expedient to present the process as well as the development and testing for the operation of methods of safe control of the object using the basics of game theory. To take into account both possible object strategies to avoid collisions, as well as their movement dynamics described by differential equations, the most appropriate model is the differential game of multiple objects (Figure 3).

State Equations
By applying the work in References [51,52], the state equations of the differential game will take the following form: (5) and by the following detailed state equations: Description of state and control variables is presented in Table 1. course of the j-th object ψj x j,4 speed of the j-th object Vj ⃗ , own object control σ0 -dimensional vector u 0,1 own object rudder deflection reference angle α r u 0,2 own object main propeller reference rotational speed n r u 0,3 own object main drive screw reference stroke H r ⃗ , j-th object control σj-dimensional vector u j,1 angular speed of the j-th object of return  u j,2 thrust force of the propeller of the j-th object F tj The basic model of differential game is a generalized model of the described control process and contains measurable and non-measurable variables. We can measure all variables on our object, but not all variables of the meet object. Therefore, variables such as H r ,  , and F tj are only found in the generalized base model. Moreover, for the synthesis of practical control algorithms, we will have to simplify this generalized model and find the surrogate models presented in Section 4 to use only measurable quantities.

Limits of Control and State
The obligation to observe the safe distance between objects, formulated by the relationship (4), leads to the following restriction on the control and state of the process: These restrictions in sea navigation are called domains that can have different shapes, for example, a circle, ellipse, hexagon, or parabola. An artificial neural network can be used to create them.

Control Criterion
As a criterion for determining the best game control out of many acceptable object strategies, the value of the so-called mathematics of game theory, integral payoff and final payoff are used: (8) When considering the work by Reddy et al. [53], the integral payoff of the game represents the closing of the own object path passing of j-th object and the final payoff of the game consists of the risk of collision r jf and descent ε f of the own object from the set route.

Computer Simulation
In the example of a container ship with a load capacity of 15,000 DWT, the state equation (6) was modeled in MATLAB/SIMULINK software, obtaining a computer simulator of controlling its own object in situations where many objects are encountered. In this way, it becomes possible to both perform a laboratory study of various possible states of the anti-collision process and test practical algorithms for the safe control of the object. Figure 4 illustrates the nature and extent of changes in state and control variables during simulation of the basic model of the game control process during an anti-collision maneuver with a change of course. According to References [54][55][56], the base model of the differential game is a computer tool for studying the movement of objects in collision situations and is used to test the software developed for navigator decision support. In addition to the high complexity of this model, simplified models are formulated for the design of practical computer control programs, with the simultaneous application of selected artificial intelligence methods.

Positional Game Model
Based on Nisan et al. [57], it can be assumed that the positional game model includes the dependency of own object strategy in encountered object position p(t k ). However, the strategies of the encountered objects are also assigned to this position.
Thus, the coordinates of the position of the objects make up the state of the entire process of their safe passing:

Sets of Object Safe Strategies
Acceptable game control of the own object u 0 and u j of the j-th encountered object includes the kinematics of their movement, recommendations of COLREGs sea route rules described in [58,59] and condition (4)-maintaining a safe passing distance D s -to create sets of acceptable game positional strategies, i.e., U 0,j and Uj ,0 : (10) Figure 5 shows the method used to determine the acceptable strategy areas resulting from the possibility of passing ships from the portside (PS) or starboard side (SS). The set of own object safe strategies U 0,j is described as follows: (11) where the individual amounts are: (12) The value of variable λ 0,j is determined using logic function L j , which characterizes the requirements of the COLREGs rules. This function results from the semantic interpretation of the text of COLREGs legal provisions in order to include them in the computer algorithm to control the movement of the object: (13) Each particular type of the situation involving the approach of the ships is assigned the logical variable value equal to one or zero: A-encounter of the ship from bow or from any other direction, B-approaching or moving away of the ship, C-passing the ship astern or ahead, D-approaching of the ship from the bow or from the stern, E-approaching of the ship from the starboard or port side [58,59].
By minimizing logical function L j by using a method of the Karnaugh's Tables following is obtained: (14) Similar to the previous dependencies, the set of safe strategies U j,0 for the j-th object is described as follows: (15) where the individual amounts are: (16) The safe trajectory of the own object, relative to all the J objects passed, is calculated by taking into account the following combined set of possible strategies:

Positional Game Control
The calculation of the optimal and safe game control of the movement of own object takes place in the following three steps: Step 1: Sets of possible strategies for individual U j,0 encountered objects in relation to own ship are defined, as well as initial sets of U 0,j possible strategies of own object for these objects.
Step 2: The own object steering u 0,j and j-th object steering u j,0 are determined, followed by the optimal positional strategy u 0 *[p(t k )] of own object, ensuring the control quality index I * optimal value.
Step 3a: For a non-cooperative positional game, the control quality index will take the form: (18) where L means the distance of own object at time t 0 to turn to the position on the reference cruise route.
First of all, from the condition of min1, the control u 0,j of the own object is established to ensure the shortest way of passing each j-th object. Secondly, with the max2 condition for the noncooperative control u j,0 of each encountered object, contributing to the largest deviation of the own object trajectory. Thirdly, from the min3 condition from the total U 0 of the own object control set to all encountered objects, the optimal control of own object is selected, ensuring the smallest path loss.
To calculate the optimal control of the own object, which ensures the three above-mentioned optimization conditions (min3, max2, min1), the use of the triple SIMPLEX linear programming method is proposed.
In this optimization task, the smallest deviation of the own object from a reference cruise route is matched by providing the maximum longitudinal component of the velocity vector of the own object from a possible set of strategies. The above calculations are repeated sequentially at each subsequent stage of the cruise route, obtaining optimal safe course ψ and speed V of own object.
Step 3b: For the cooperative positional game, the control quality index will take the form: (19) The difference from the non-cooperative game algorithm is that the max2 criterion is changed to the min2 criterion.
Computational programs of the pgnc non-cooperative positional game, according to (18), and the pgc positional cooperative game, according to (19), were implemented in the software (lp) linear programming from the Optimization Toolbox MATLAB.

Computer Simulation
The pgnc and pgc algorithms have been subjected to simulation tests in SIMULINK software, in situations where the own object passes 13 other objects whose data from the radar remote sensing ARPA system is listed in Table 2 and shown in Figure 6.

Matrix Game Model
Replacing the differential equations of dynamic motion of objects with the appropriate maneuver advance time allows the differential game model to be brought into the surrogate matrix game model.
Then, the objects controlling their movement by means of course and speed changes, and their current state, related to traffic safety, is characterized by mutual distance D j and bearing N j with the radar remote sensing ARPA system and r j risk of collision.

Matrix of Risk Collision
Having considered the work by Modarres [60], the author of this article proposes a new formulation of object collision risk as a reference to two assessments of the same navigational situation.
The first assessment contains the parameters D j min and T j min of the real situation of the proximity of the objects. The second assessment concerns the same situation, but the safety is determined by parameters D s , T s, and D j .
Thus, this reference has three relative elements: D j min /D s , T j min /T s, and D j /D s , all of which are proposed to express the risk of collision r j (u 0 ,u j ) as the following mean square form of these three relative elements: (20) where the weighting factors η 1 , η 2 and η 3 assess the state of visibility at sea-good or limited type of navigation area and open or fairway (Figure 9). Matrix game R in which player OO-own object can use u 0 clean strategies, and player EOencountered objects with u j clean strategies, which can be described using the following collision risk matrix: The number of rows in the R matrix risk of collision is equal to the number of possible OO strategies of the player as deviations of his course or speed: (22) The number of columns consists of the total number of permissible strategies for all the players EO involved in a collision situation, by analogy to all the changes in the course and speed of each met object: (23) In addition, we need to follow the COLREG rules, which impose restrictions on the possible strategies (u 0i , u ji ).

Matrix Game Control
In practice, the matrix game of many objects only uses pure strategies that do not reach the state determined by Nash [45] equilibrium points. Then, a substitute solution to such a real game is obtained by using mixed strategies that are the probabilities of pure strategies.
Step 1: The probability matrix P of using pure strategies in the game control will take the following form: (24) The optimal control of the own object is provided by its strategy, which has been assigned the highest probability of use: (25) Step 2a: The optimal and safe own object steering in the non-cooperative matrix game has the following dependence: (26) First, from the max1 condition, it is assumed that the j-th object control u j contributes to the largest deviation of the own object's trajectory. Then, from the min2 condition, for these control u j , own object control u 0 is assumed, providing the shortest safe way of passing each j-th object.
Step 2b: The quality control index of optimal and safe own object steering in the cooperative matrix game will take the following form: (27) The difference from non-cooperative game algorithm is that the max1 criterion is changed to the min1 criterion.
Computational programs of the mgnc non-cooperative matrix game, according to (26), and the mgc matrix cooperative game, according to (27), were implemented in the software (lp) linear programming from the Optimization Toolbox MATLAB.

Computer Simulation of the Matrix Game Model
The mgnc and mgc algorithms have been subjected to simulation tests in SIMULINK software in situation examples of own objects passing 13 other objects whose data from the radar remote sensing ARPA system is listed in Table 2. Simulation studies that reflected the movement of objects, both in good visibility at sea for D s = 0.5 nmi (nautical miles) and in limited visibility at sea at D s = 2.0 nmi (nautical miles), are shown in Figures 10 and 11. Additionally, it should be clarified that, in the drawings, the trajectories of the objects encountered do not change due to the difficulty of presenting them based on complicated operations while determining the optimal element of risk matrix R and/or the probability matrix P. Figure 10. Object game trajectories calculated according to the mgnc algorithm of non-cooperative matrix game control: on the left-with good visibility at sea, with the final payoff of the game ε f = 9.2 nmi achieved, assessing deviation safe trajectory from reference cruise route; on the right-with limited visibility at sea, with the final payoff of the game ε f = 13.8 nmi.

Comparison of Game Object Control Methods
The results of computer calculations of the safe trajectory of the own object in the situation of passing by 13 encountered objects, made using developed game algorithms, are presented in Figure  12. The deviation of the safe trajectory from its initial direction, with good visibility at sea, ranges from 0.7 nmi for a cooperative positional game, to 9.2 nmi for a non-cooperative matrix game.
However, this is a deviation from the trajectory in limited visibility at sea, which ranges from 2.7 nmi for a cooperative positional game to 13.8 nmi for a non-cooperative matrix game.

Conclusions
An analysis of the causes of collision of objects at sea, especially in situations where multiple objects pass each other and in limited visibility at sea, indicates the importance of the navigator's subjectivity in making steering decisions, often leading to conflict situations. Game theory comes to the rescue as a theory of resolving these conflict situations. This approach allows for the development of a more adequate model of the anti-collision process, in the form of a multi-object differential game method.
The differential game method, as a very accurate multidimensional process model, is most useful as a laboratory simulation model for the synthesis and research of practical quarterback control algorithms. For the design of such algorithms, surrogate models using positional and matrix methods, which perform a computer-assisted navigator role and significantly accelerate in making optimal and safe decisions, are the best suited.
These algorithms take into account both the COLREG rules when starting the game and the dynamics of the objects in the form of the maneuver advance time, their degree of cooperation, and the end game if the risk of collision becomes zero.
The essence of the article is to use elements of game theory to consider navigator subjectivity in collision situations and surrogate game models as positional and matrix games in the design of the entire calculation steering algorithms. A new definition of object collision risk was also presented as a reference to two assessments of the same navigational situation: the real situation of the proximity of objects and the safe situation determined by the reference parameters.
However, the designated set-up trajectories consist of consecutive maneuvers, major course changes, the smoothness of the calculated object path is important, to which the works of Hu et al. [61], Campbell et al. [62], and in particular Ravankar et al. [63] are devoted.
This work did not exhaust all the issues related to safe steering of objects at sea. In future projects, consideration of safe object steering sensitivity analysis to the data inaccuracy of ARPA radar remote sensing system, changes in object kinematics and dynamics parameters, and the impact of outside disturbances could be included.
The article deals with the deterministic approach to the problems of games and their application to control multiple moving objects. In the next works, one can consider elements of the theory of stochastic games and the possibilities of their application. Statistical evaluation of the proposed methods would also be important by simulation testing of typical navigation scenarios, for example on the ARPA simulator.