Dynamic Games Methods in Synthesis of Safe Ship Control Algorithms

The paper presents application of dynamic games methods, multistage positional and multistep matrix games, to automate the process control of moving objects, on the example of safe control of own ship in collision situations when passing many ships encountered. Taking into consideration two types of ships cooperation, for each of the two types of games, positional and matrix, four control algorithms for determining a safe ship trajectory supporting the navigator’s maneuvering decision in a collision situation are presented.The considerations are illustrated by examples of computer simulation in Matlab/Simulink software of safe trajectories of a ship in a real situation at sea. Taking into account the smallest final deviation game trajectory from the reference trajectory of movement, in good visibility at sea, the best is trajectory for cooperative matrix game, but in restricted visibility at sea, the best is trajectory for cooperative positional game.


Introduction
One of the most important transport issues are the processes of optimal and safe control of ships, airplanes, and cars as moving objects [1][2][3][4].Such processes relate to managing the movement of many objects at the same time, with varying degrees of interaction, the impact of random factors with an unknown probability distribution, and a large share of operator's subjective in maneuvering decisions [5][6][7][8].Therefore, the management of such processes is accomplished by means of game control systems, whose synthesis is carried out with the methods of game theory [9,10].Game theory is a branch of mathematics, covering the theory of conflict situations and building and analyzing their models [11,12].Conflict can be as follows: military, political, social, and economic, in a social game, in the game with nature, and in the implementation of the control process during interferences of disturbances or other control objects [13].A game in the concept of control theory is a process consisting of several control objects remaining in a conflict situation or a process with undefined disturbances or incomplete information.Players as control objects participating in a conflict situation have certain sets of strategies.Strategy is a set of rules of action, player control, which cannot change the actions of an opponent or nature [14][15][16].The strategies are implemented by man, automaton, regulator, and computer.Strategies can be pure as elements of a set of strategies or mixed as a probability distribution on a set of clean strategies.The result of the game is the payment in the form of winning, losing, or the probability of carrying out a certain action-control [12,[17][18][19].
The first concept of game theory and the theorem on mini-max was formulated by E. Borel (1921Borel ( , 1927)).The main creators of game theory are John von Neumann (1928) and O. Morgenstern (1944).
The largest class of games that can be used in the game control of dynamic transport processes and among them controlling the movement of ships, planes, and cars represent differential games, described by state and output equations, and state and control constraints [20][21][22][23][24].

Classification of Control Processes of Moving Objects
As a result of the movement of own ship with speed  and course  in terms of encountered  ship moving at a speed   and course   a situation at sea is determined.Parameters characterizing the situation as distance   and bearing   for  ship are measured by radar anticollision system ARPA (Automatic Radar Plotting Aids) [25].
The ARPA system enables us to track automatically at least 20 encountered  ships, determine of their movement parameters (speed   , course   ) and elements of approach to the own ship (  min =   : Distance of the Closest Point of Approach,   min =   : Time to the Closest Point of Approach), and also assess the collision risk (see Figure 1).
The proper use of anticollision system ARPA in order to achieve greater safety of navigation requires, in addition to training on the use and interpretation of the data, supplements the system with appropriate methods of computeraided maneuvering decision of navigator in the complex navigational situation in a short time, eliminating the subjectivity of man and taking into account the indefiniteness of the situation and the properties game process control [26,27].
In practice, there are many possible maneuvers to avoid a collision, from which to select the optimal manoeuver, to ensure a minimum the risk of collision or minimum losses of the road for safe passage of encountered ships (see Figure 2).The movement of objects in time is influenced by control variables  from appropriate admissible control sets : where  () 0 set of strategies for own object,  ()  set of strategy  of encountered object from among the total number of  objects,  = 0 symbolically means stabilization of the set object trajectory,  = 1 symbolically the implementation of an anticollision manoeuver to minimize the risk of collision, which in practice is achieved by meeting inequalities: min : smallest distance of approaching own object to the object you are meeting,   : current distance to the object ,   : safe proximity distance in given ambient conditions, traffic rules, and dynamic properties of the object,  = −1: symbolic maneuvering the object in order to achieve the shortest approaching distance, for example, when transferring the load.

Dynamic Games Models of Moving Objects Control Processes
3.1.Differential Game Model.The most adequate model of the process of controlling own object in the situation with  encountered objects is the differential game model of  participants (Figure 3).The dynamic properties of control process are described by the state equation: where For example, the equations of the state of the own ship control process in collision situations, taking into account the own ship's hydromechanics equations and the kinematics of relative motion of own ship and ship encountered, will take the form (4).   : linear velocity.For example, for the passing situation of own ship with  = 20 ships encountered, the differential game model of this process is represented by  = 86 state variables [28].
The constraints of state and control variables result from the fact that ship maintains a safe passing distance   in accordance with the COLREGs (Collision Regulations) rules of maneuvering with each ship encountered [29,30]: The synthesis of game control object consists in minimizing the control quality index in form of integral and final payment: If the velocity of own object variable is assumed, then the integral payment will show the length of the trajectory of its own object during the passing of objects encountered.The final payment determines the final risk of collision of own ship to the th object and the final deviation of own object trajectory from the previously reference trajectory of traffic [2,6,8].The advantage of the game model is an accurate description of the kinematic and dynamic properties of the process of controlling objects.The disadvantage of this model is the large number of state variables and the complexity of mathematical dependencies.Therefore, this model is best used as a simulation model for testing practical algorithms for controlling the safe movement of objects [31][32][33][34][35][36].
For the purposes of synthesis of real control algorithms, simplified game models in the form of positional and matrix games are used.

Journal of Advanced Transportation
It is assumed that at each discrete moment of time the own ship position and the met objects positions are known.Constraints of state variables are navigational constraints on the surrounding of encountered objects: Constraints of control variables take into account the motion kinematics of objects, legal recommendations for traffic regulations (maritime traffic law, air traffic law, and road code), and the condition of maintaining a safe passing distance: The sets of acceptable strategies of the players in relation to each other are dependent, which means that the choice of control by the th of the object changes the sets of acceptable strategies of other objects: The resultant area of acceptable manoeuvers of own object in relation to  objects is Optimal control of the own object, ensuring minimal road loss on safe passing of the objects encountered, is determined by the static optimization method from the set of permissible controls  0 : Constraints on the choice of strategy ( 0 ,   ) result from legal recommendations of traffic COLREGs regulations.Because usually the game has no saddle point, so there is no guaranteed balance [37,38].

Game Ship Control Algorithms
The synthesis of algorithms for the control of moving objects was carried out on the example of the safe motion control process of one's own ship during the meeting of other ships.Individual models of the process can be assigned the appropriate algorithms of computer-aided navigating maneuvering decisions in collision situations [39,40].
The exact but complex model of differential game serves as a simulation model to check the correctness of control algorithms based on approximate positional and matrix game models.

Algorithm of Positional Noncooperative
Game.The optimal control of your own ship is calculated by determining the sets of acceptable strategies for the ships you meet with respect to own ship and the sets of acceptable own ship strategy for each of the ships you meet.Then the optimal positional strategy of the own ship is determined from the condition: The goal control function of own ship  0 characterizes the distance of own ship to the nearest point of return   on a given voyage route.The criterion for choosing the optimal trajectory of own ship is to determine its course and speeds ensuring the smallest loss of the path for safe passing of encountered ships, at a distance not lower than the assumed value of   , taking into account the dynamics of own ship in the form of advance time of manoeuver.First, the control of own ship is determined to ensure the shortest trajectory of the flight, the smallest loss of the road (min condition) for noncooperating control of every ship encountered, contributing to the largest extension of the trajectory of the own ship (max condition).At the end, from the set of controls of own ship to particular  placed ships, the control of own ship is selected in relation to all  ships encountered, ensuring the smallest loss of the road (condition min).According to the optimization three conditions (min max min), the linear programming method is used to solve the game, obtaining the optimal values of the course and the speed of own ship.The smallest road losses are achieved for the maximum projection of the ship's own speed vector on the course direction.Optimal control is calculated many times at each discrete stage of motion using the SIMPLEX method to solve the linear programming problem for variables in the form of components of the ship's own speed vector [41].

Algorithm of Positional Cooperative
Game.For a cooperative game, the control criterion (15) will take the following form: The difference in relation to the previous algorithm results from the cooperation in avoiding collision by all objects encountered  and replacing the second condition max for min [42].

Algorithm of Matrix Noncooperative Game.
The dual linear programming method can be used to determine the optimal control.In the dual issue, player A seeks to minimize the risk of collision, while player B in the noncooperative game aims to maximize the risk of collision [43,44].The components of the mixed strategy express the probability distribution of players using their pure strategies.As a result, for the control criterion in the form a matrix of probabilities of using individual pure strategies is obtained.
The most secure probability of   is the solution to the task of safe control of own ship: Applying dual linear programming to matrix game solution, the optimal values of own ship course and th met ship are obtained, with the smallest deviations from their initial values.

Algorithm of Matrix Cooperative
Game.For a cooperative game, the control criterion (17) will take the following form: The difference in relation to the previous algorithm results from the cooperation in avoiding collision by all objects encountered  and replacing the second condition max for min.The game ends at the moment   , when the risk of own ship   in relation to each  ship will reach the value of zero   (  ) = 0 and then the final deviation of the trajectory of own ship from reference trajectory (  ) is assessed.

Computer Simulation of Game Ship Control Algorithms
Figure 8 compares the trajectories calculated by individual four algorithms.
In Figure 8(a), showing the safe trajectories of own ship in conditions of good visibility at sea, the best is trajectory 4 for a cooperative matrix game, providing the smallest final deviation from the reference trajectory of movement, (  ) = 0.57 nm.
In Figure 8(b), showing the safe trajectories of own ship in conditions of restricted visibility at sea, the best is trajectory 2 for a cooperative positional game, providing the smallest final deviation from the reference trajectory of movement, (  ) = 1.56 nm.

Conclusions
The use of simplified differential game models of the control process of moving objects, in the form of a multistage positional game and multistep matrix game, for the synthesis of control algorithms allows us to determine the safe optimal and game trajectory of own object in passing situations with more objects as a sequence of manoeuvers at a course and speed.The developed control algorithms take into account the legal rules of object movement and manoeuver advance time, approximating the dynamic properties of the own object and assessing the final deviation of the actual trajectory from the reference one.The presented control algorithms constitute formal models of the actual decision-making processes of the ship's navigator and can be used in the computer navigator support system when making manoeuver decisions in collision situation.

Figure 2 :
Figure 2: Possible trajectories of the own ship in case of passing the encountered ships.

Figure 3 :
Figure 3: Diagram of the differential game model of moving objects control process.

3. 2 .
Positional Game Model.The differential game model comes down to a multistage positional game model, in which the object's dynamics are taken into account by the time of the manoeuver ahead of time.The essence of positional game is dependence of own object's strategy on the positions of the objects () taught.In this way, possible changes in the course and speed of the objects encountered during the control implementation are taken into account in the process model.The current state of the process at the moment   is determined by the coordinates of own ship position  0 and encountered objects   :  (  ) = [  0 (  )   (  ) ] ;  0 = (  ,   ) ;   = (  ,   ) ; ( = 1, 2, . . ., ) ; ( = 1, 2, . . ., ) ,

Figures 4 , 5 , 6 ,
Figures 4,5,6,and 7 show the own ship safe trajectories determined by four algorithms previously in the MAT-LAB/SIMULINK software, in the situation of  = 34 encountered ships in the Kattegat Strait, in conditions of (a) good visibility at sea for   = 0.3 nm (nautical miles) and (b) restricted visibility at sea for   = 1.5 nm.The game ends at the moment   , when the risk of own ship   in relation to each  ship will reach the value of zero   (  ) = 0 and then the final deviation of the trajectory of own ship from reference trajectory (  ) is assessed.Figure8compares the trajectories calculated by individual four algorithms.In Figure8(a), showing the safe trajectories of own ship in conditions of good visibility at sea, the best is trajectory 4 for a cooperative matrix game, providing the smallest final deviation from the reference trajectory of movement, (  ) = 0.57 nm.In Figure8(b), showing the safe trajectories of own ship in conditions of restricted visibility at sea, the best is trajectory 2 for a cooperative positional game, providing the smallest final deviation from the reference trajectory of movement, (  ) = 1.56 nm.

Figure 4 :
Figure 4: Safe trajectory of own ship in the situation of passing with  = 34 met ships, determined by the positional noncooperative game algorithm.

Figure 5 :Figure 6 :
Figure 5: Safe trajectory of own ship in the situation of passing with  = 34 met ships, determined by the positional cooperative game algorithm.

Figure 7 :
Figure 7: Safe trajectory of own ship in the situation of passing with  = 34 met ships, determined by the matrix cooperative game algorithm.

Figure 8 :
Figure 8: Comparison of safe trajectories of own ship in the situation of passing by with  = 34 met ships, determined by individual four algorithms: (1) positional noncooperative game, (2) positional cooperative game, (3) matrix noncooperative game, and (4) matrix cooperative game; (a) conditions of good visibility at sea for   = 0.3 nm and (b) conditions of restricted visibility at sea for   = 1.5 nm.