Distributed Cooperative Search Control Method of Multiple UAVs for Moving Target

To reduce the impact of uncertainties caused by unknownmotion parameters on searching plan of moving targets and improve the efficiency of UAV’s searching, a novel distributed Multi-UAVs cooperative search control method for moving target is proposed in this paper. Based on detection results of onboard sensors, target probability map is updated using Bayesian theory. A Gaussian distribution of target transition probability density function is introduced to calculate prediction probability of moving target existence, and then target probability map can be further updated in real-time. A performance index function combining with target cost, environment cost, and cooperative cost is constructed, and the cooperative searching problem can be transformed into a central optimization problem. To improve computational efficiency, the distributedmodel predictive control method is presented, and thus the control command of each UAV can be obtained. The simulation results have verified that the proposed method can avoid the blindness of UAV searching better and improve overall efficiency of the team effectively.


Introduction
In recent years, Unmanned Aerial Vehicles (UAVs) have been widely used in both civilian and military fields, performing a variety of missions such as searching, reconnaissance, and surveillance [1].Carrying a series of sensors, they can search kinds of targets in mission areas and obtain large amounts of useful information.Currently, searching method is generally preplanning trajectory method, and it is effective to search steady and nonthreat targets.However, with the complexity of the environment and increase of threats, it will become very difficult for a single UAV to accomplish the tasks of searching and capturing.It may be destroyed by the enemies, and then the entire mission will be in failure.To solve this problem, many scholars have proposed cooperative search method by multiple UAVs [2][3][4].Multi-UAVs cooperative search is precondition performing other tasks such as target tracking, attack, and evaluation.Efficient searching can find the targets in mission areas and obtain intelligence information.Then, how to realize better cooperation between UAVs has become an important problem to be resolved.
For the problem of Multi-UAV cooperative search, there are many literatures developed by associated scholars.In [5,6], a cooperative search method based on distributed model predictive control (DMPC) is presented.Based on this method, a centralized online optimization problem is decomposed into decentralized optimization problem of each UAV.In [7], the objective of teams is proposed to search and cover the whole of the unknown area, while avoiding collision.Another problem is communication delay between UAVs when carrying on cooperative search in an uncertain environment [8,9].In [8], a stochastic method is proposed to estimate the probability of different actions for each UAV, and it can effectively compensate the effect of communication delay.For searching static targets, -shortest path algorithm is introduced to guide the UAVs in an unknown area.Compared with stochastic and greedy searching algorithm, it is verified to be more effective [10].In [11], the distributed approach integrating dynamic programming is proposed, and the objective is to gain more information of the whole team.By this method, each UAV can find its own flight path, respectively.Although these above methods have been verified to be effective for cooperative searching problems, they are mainly about the static targets, not suitable for the moving and time-critical targets.Currently, searching moving targets is the main focus of the satellites and robots fields [12][13][14][15][16][17][18].
Here several typical results are listed.In [12], five methods are proposed to search the ocean moving targets, and their performances are compared by simulation experiments according to creditable orbit data of the satellite.In [14], a swarm of UAV is used to search one or more evading targets in a predefined area.These targets have the abilities of avoiding the detection of the UAV.By negotiation between UAVs, the swarm can optimize their sensing capabilities to ensure the maximal territory of searching.In [16], for the uncertainty of target motion, the distributed target-centric formation control strategy integrating sliding mode control is proposed.It can compensate the uncertainties and maintain the expected target-centric formation.
In this paper, we focus on searching the moving targets using Multi-UAV cooperation.Our main contributions contain three aspects: (1) Cognitive information map is constructed to describe the interaction among target, environment, and UAVs.(2) A novel updated mechanism for cognitive information map is designed integrating the prediction of target motion.(3) The pheromone map and artificial potential field are combined into the cooperative cost, which can ensure effective cooperation and collision avoidance between UAVs.
The remainder of this paper is organized as follow.In Section 2, the problem description is presented.The model of cognitive information map is built in Section 3. Based on these efforts, the performance function of UAV searching is constructed, and the receding horizon optimization method is used to solve distributed cooperative decision-making problem in Section 4. The simulation results are shown in Section 5.In Section 6, we conclude this paper and point out the further direction of this work.

Problem Descriptions
2.1.The Description of Search Environment.As is mentioned above, UAVs detect the ground moving targets by using onboard sensors with a fixed angle.So the whole environment can be mapped into a two-dimension space.Given a mission area  ∈  2 , suppose field of view (FOV) of the sensor to be approximately as a square area.Using geometric method, the mission area can be divided into   ×   same grids.Let the coordinate (, ) denote the position of one grid, where  ∈ {1, 2, . . .,   } and  ∈ {1, 2, . . .,   }.It is supposed that each grid can store some useful information, containing the existing probability of target, the uncertainty degree of environment, and the situation.Combining with all the grids, the cognitive information map (CIM) for searching can be obtained.
To better describe the searching problem, three hypotheses are given as follows: (1) There is only a target in each grid at the same time.(2) Communication between UAVs has no delay or interruption.(3) There are no threats and obstacles in the environment, and only take into account the collision problem between UAVs.

Simplified Dynamic Model of UAV.
Assume that each UAV has a certain speed and orientation.Let (  (),   ()) and   () = {0, 1, 2, . . ., 7} denote the position and flight orientation of UAV , respectively.There are eight possible flight orientations, shown in such an array {0 (North), 1 (Northeast), 2 (East), 3 (Southeast), 4 (South), 5 (Southwest), 6 (West), 7 (Northwest)}.The UAV's dynamics is subjected by its curvature radius constraints.That means it can only change its orientation at most once by one step, namely,   (+1) ∈ {  ()−1,   (),   ()+1} mod 8.Each UAV has three possible orientations in the next time step, such as turn left, go straight, and turn right.It means that UAV's maximum turning angle is 45 ∘ .For each UAV, the decisionmaking is which path will be selected in the next time step.Take UAV , for example; let   () ∈ {, , } denote its selected path at each time step . Figure 1 shows the schematic diagram for searching environment and various orientations.
As is shown in Figure 1, it is composed of two schematic diagrams.The left one is the description of UAV's searching environment.The red triangle denotes UAV, the blue circle denotes target, and the black rectangle region denotes a preassumed river or lake, where there is a lower probability or even zero.The green rectangle region denotes a camp or an interesting target region, where there is a higher probability.The right one shows the possible flight orientations of UAV at the current and the next time step.If UAV flies towards the north at time , there will be   () = 0.For each UAV, there are three possible selections for flight orientations at time  + 1, which are   ( + 1) = 7 (towards northwest),   ( + 1) = 0 (towards north), and   ( + 1) = 1 (towards northeast), respectively.But only one orientation can be selected, which should be most useful for searching based on acquired external environment information.

The Model of Cognitive Information Map
Traditional existing probability map (TEPM) constructed in Section 3.1 only considers target existing probability in the task area, but it is not enough for Multi-UAVs cooperative search moving target.Next, the other two maps are introduced on the basis of TEPM.One is called the uncertain map of environment (UEM), and it can describe the certain degree of unknown environment which has been visited by UAVs.The other is called the pheromone map (PM), which is mainly used to establish the cooperation mechanism between UAVs and the controllable accessing ability to environment.So, integrating TPM, UEM, and PM, the cognitive information map (CIM) can be designed.

The Uncertain Map of Environment.
For each grid cell, a number is set, which denotes uncertain degree of environment.For example, let (, , ) denote uncertain degree of grid (, ) at time , and let  denote descent factor.When UAV flies across the cell and understands grid cell, uncertain degree of grid cell will decrease.Set a certain threshold for the uncertain degree, and it means UAV completely understands the environment.Using the following equation, uncertain degree of grid cell can be updated: (, , ) , others. (1)

Pheromone Map.
In order to realize the effective cooperation between UAVs, two basic pheromones [19] which are attractant and repulsion are introduced in this section.For these two pheromones, two mechanisms are introduced, namely, propagation and evaporation.Define   and   as the coefficients of attractant and repulsion propagation, respectively, while defining   and   as the coefficients of attractant and repulsion evaporation, respectively.Similarly, define two same pheromones for each grid.Let   (, , ) and   (, , ) denote attractant and repulsion pheromone of grid (, ) at time , respectively.
Assume that  , is the last accessed time of the grid cell (, ),  0 is time threshold which is accessed again, and  , is switch coefficient of attractant pheromone of grid cell (, ).If  −  , ≤  0 , then  , = 0, which means grid (, ) is not be accessed again.If  −  , >  0 , then  , = 1, which means grid (, ) should be accessed again.The switch matrix of attractant pheromone is calculated as follows: Assume that   is the accessed state matrix, storing the state of each grid accessed.Let  , denote the times of UAV's accessing grid (, ) in the last period, where  , ≥ 0. Consider Then, the attractant strength of the current grid can be calculated by (5), shown as follows: where   (, , ) denotes attractant strength of the grid (, ) at time ,   (, ,  + 1) denotes the release amount of attractant pheromone at time  + 1, and   (, ,  + 1) denotes attractant pheromone from neighboring grid between times  and  + 1.
Similar to attractant pheromone, the repulsion pheromone of current grid (, ) can be calculated by (6).Consider where   (, , ) denotes repulsion strength of the grid at time ,   (, ,  + 1) denotes the release amount of repulsion pheromone at time  + 1, and   (, ,  + 1) denotes repulsion pheromone from neighboring grids between times  and +1.

The Existing
Figure 2: The potential movement area of maneuvering target.
know the target distribution states.Existing probability map [11] is used to describe target existing probability of each grid.
Let (, , ) denote target existing probability of grid (, ) at time , let   denote sensor detection precision, let   denote sensor false probability, and let () denote whether the target is detected or not at  time.If () = 1, it means UAV has detected target, while () = 0 means that UAVs do not detect target.Using the onboard sensors to detect all the grid cells, target existing probability is updated according to the detecting results.The updated mechanism is as follows: (, ,  + 1) To reduce the uncertainty of target motion parameters and improve the searching efficiency, prediction model of targets motion is established in this section.Then, a Gaussian distribution transition probability density function of moving target is proposed.Based on updating target existing probability in the TEPM, the existing prediction probability of the target can be calculated to further update the TEPM.So it can avoid the searching blindness and improve cognition and prediction ability for the target.

The Description of Potential Movement Area of Maneuvering Target.
If the prior intelligent information and the previous uncertain motion of maneuvering target manners are not accurate, it will be very difficult for UAV to search maneuvering target.Therefore, it is very important to predict the motion manners of target.Here, the impact on these constraints is considered.The potential movement area of maneuvering target is built, shown in Figure 4. Assume that UAV has acquired prior information at time  and enters search area after Δ.During this time, target is moving.Suppose, at time  0 , the position of maneuvering target is ( 0 ,  0 ),  denotes velocity of target,  denotes heading angle of target, Δ max is the maximum heading angle, and Δ max is the maximum velocity.The potential motion area of maneuvering target in time Δ is described as such a region, where ( 0 ,  0 ) is center, (V + V 0 )Δ and (V − V 0 )Δ are radius, and the shadow between heading angle ( −  0 ) and ( +  0 ) is target potential area, shown in Figure 2(a).If prior information is deficient, there will be (V − V 0 ) ≈ 0 and ( +  0 ) − ( −  0 ) ≈ 360 ∘ .It means velocity and heading angle of target are unknown; then the potential movement area of maneuvering target can approximate a cirque, shown in Figure 2(b).

The Motion Prediction Model of Maneuvering Target.
In this paper, the probability distribution of maneuvering targets in potential movement area is used to predict the motion manners of target.Assume  −1 and  −1 denote position component of target on the axis of  and  at time  −1 , respectively.Similarly,   and   denote position component of target on the axis of  and  at time   .Here, three hypotheses are given.(1) The position component of target on the axis of  and  is independent.(2) There is a larger time interval between  −1 and   , and  denotes the number of time step, where Δ = Δ.(3) The acceleration is the same in each time step and subject to Gaussian white noise sequence   with the same variance.
According to the above description, mathematical induction method can be applied to gain the position and velocity of the target at subsection  between  −1 and   time, respectively.Consider According to formulation ( 7)-( 10), suppose  = .In this paper,  = 10 is selected.Then, the position and velocity of the target towards and -axis at time   can be obtained as follows:

The Transition Probability Density Function of Maneuvering
Target.The uncertainty of target position is mainly determined by the moving direction of the target.After Δ, the circle with the center ( −1 ,  −1 ) and the radius VΔ is formed.The transition probability density function of maneuvering target can be obtained.It is subject to a Gaussian distribution, the mean values of which are the points on the cycle mentioned above.The schematic of transition probability density function of maneuvering target is shown in Figure 3.
Assume that the position of target is ( −1 ,  −1 ) at time  −1 , after Δ, and the position of target along and -axis will be such Gaussian distributions with the means   and   , variance  2 , respectively.According to formulation (13), solving the mean and variance of random variable   , it can be deduced as follows: Combining ( 17) with ( 16), the variance of the random variable   can be obtained: Assume that the estimation of moving velocity is V and the angle  is between the velocity direction and -axis, and then (15) can be transformed as follows: Similarly, one can obtain the mean   and variance  2 of random variable   : Assume random variables   and   are independent.After Δ, the target moves from ( −1 ,  −1 ) to (  ,   ), and then transition probability density function will be (22)

The Existence Prediction Probability of Maneuvering
Target.For ground moving target, UAV continuously needs to detect, update, and predict the moving of target and makes use of predictive result to update the existence prediction probability of the target.The predictive result will be the prior information for UAV's decision-making at next time step.Assume that the probability of the target existing in the grid  at time  −1 is   ( −1 ).The length and width of the grid are both .After Δ, UAV will predict the moving situation of targets and gain the transition probability density function of the target moving from grid  to grid .Consider The existence prediction probability of targets in the grid  at time   is shown as follows: p (  ) =  (, , Δ)   ( −1 ) . (24) At last, the cognitive information map (CIM) integrating uncertain map of environment, probability map of target, and pheromone map is completely built.It can provide more information to UAVs and ensure the effective cooperation between Multi-UAV.

The Decision-Making Approach of Distributed Cooperative Search of Multi-UAVs
In order to ensure effective cooperation and avoid collision between UAVs, the special mechanism based on the combination of pheromone map and artificial potential field method is established.Meanwhile, the correlative value function is constructed.But, for centralized model prediction control, calculation amounts are very large.If the central UAV is disabled, the other UAVs will not acquire decision-making commands; thus the whole searching task will be failed.So the centralized online optimization problem of Multi-UAVs is transformed into the distributed online optimization problem.It can improve the calculation efficiency and ensure successful accomplishment of the searching task.
4.1.The Performance Index Function of UAV Searching.The purpose of cooperative search is to find as many targets as possible and to reduce the uncertain degree of the whole searching area, thus ensuring the effective cooperation between UAVs and avoiding collision [19,20].We can construct the function of searching efficiency ((), ()) to describe performance of UAV searching.This function is a multiobjectives optimization function, containing coverage, target discovery, and cooperative capability.Next, we will introduce the modeling process of the search cost of environment, the discovery cost of target, and cooperative cost in detail.
(1) The Search Cost of Environment.The search cost of environment is mainly used to describe how to reduce the uncertainty degree of environment.The information entropy is introduced to denote the uncertainty degree of environment, presented as (, ,   ).The special definition is described as follows: ( With UAV continuously searching and detecting in the mission area, the information entropy may gradually decrease.So, in order to describe the reduced amount of information entropy, the search cost of environment is defined as follows: (2) The Discovery Cost of Target.The discovery cost of target denotes the possibility of UAV finding targets based on onboard sensors from the current position to target position: where (, ,   ) is updated by formulation (2),   is the detection precision of the sensor, and   is false probability of the sensor.
(3) Cooperative Cost.The purpose of cooperation optimization is to avoid the excessive repeated searching of one grid.Therefore, the pheromone is introduced to describe the occupation situation of searching area.By this way, it can effectively reduce the numbers of UAV visiting the grid repeatedly and improve the cooperative searching efficiency.

Collision Avoidance Mechanism between Multi-UAVs.
For Multi-UAVs cooperative search problem, we not only consider cooperative search efficiency but also consider the collision and trajectory overlap between UAVs.Here, the artificial potential field method is introduced to solve the above problem.This method is a virtual method and can ensure the relative position and orientation by calculating competition force between UAVs.Let   (  ) denote the competition force between UAV  and UAV  at time   , which is defined as follows: where   denotes the shortest distance between the flight path of UAV  and UAV , if   = 0, the constant  is changed to a larger number, and if   is gradually rising,  →   (  ) is gradually dropping.
To facilitate the description, two conditions are given.
Condition 1.The distance between the UAV  and UAV  is less than the maximum distance , and the UAV  lies in flight observation scope of UAV , where the scope is a sector area which is between the maximum angle  and the minimum angle −.
Condition 2. The difference   of heading angle between the UAV  and UAV  is less than the threshold  th ; that is, The condition and the vector of competition force between UAVs are shown in Figure 6.From Figure 4, it can be found that UAV 3 and UAV 4 satisfy Condition 1, both in the flight range of UAV 1 and less than the maximum distance.UAV 2 and UAV 3 satisfy Condition 2, the angle between UAVs is less than  th , so only UAV 3 satisfies Conditions 1 and 2 at the same time, and only UAV 3 produces competition force for UAV 1; others do not produce competition force for UAV 1; that is,   →  21 = 0,   →  41 = 0, and The resultant force is the vector sum of competition forces which belong to the other UAVs act on the UAV .
As is mentioned above, both UAV's repeated access and collision avoidance between UAVs should be considered, so we will model competition force value function, shown as follows: where   (,   ) denotes angle difference between the direction of total rivaling force  →   (  ) and the direction of each possible path. 0 is a positive constant.
Combining ( 28) and (31), the cooperation value function can better express cooperative mechanism between UAVs.It can both improve the efficiency of cooperative search and avoid collision between UAVs: The total value function ((), ()) is obtained by weighting every performance function: (34)

The Decision-Making Approach of Distributed Cooperative Search.
The purpose of Multi-UAV cooperative search moving targets is to ensure maximizing search efficiency in uncertain area, obtaining the information of the whole search area, decreasing the uncertain degree of environment, and finding the maximum number of targets.Assume that the discrete states equation of Multi-UAVs cooperative search system is shown as follows: where  is the discrete time, () is the system states, () is the control input,  is the mapping from input to output, and  is the length of predictive horizon.Here, () and () contain the states and controls of all UAVs, shown as follows: Based on the current state () and decision-making sequence (,  +  − 1) during the horizon time, predictive states sequence (+1), (+2), . . ., (+) can be obtained.Assume that () and () are the sets of the states and decision-making sequence, respectively, shown as follows: Here,   () and   () denote the state and control sequence of UAV V  , respectively.Assume that ((), ())

Communication topology
Local DMPC controller where Ξ and Θ denote the feasible state set and admissible input set of UAV, respectively, Ξ  denotes state constraint set of UAV V  , and ((), ()) ≤ 0 denotes the various constraint conditions.
For centralized optimization model, the central node (UAV or ground station) is responsible for obtaining the state information of all UAVs and assigning the optimization control sequence to each UAV.This method has a good ability of global decision-making, but it may bring severe dimension disaster when the numbers of UAV and prediction horizon time increase.But, for each UAV, it must make the decision in a short time.So distributed model predictive control (DMPC) [20] method is introduced to realize decision-making of UAV searching.The information structure of UAV autonomous cooperative search based on DMPC is shown in Figure 5.
For such a dynamic decoupling system, assume that   (  V (),   V ()) is the state equation of UAV V  .UAV V  can obtain information  − () by the network communication and make the decision by the DMPC method.It also can share the state   V () and the decision   V () with other neighboring grids.For the UAV V  , the local horizon optimization (40) The distributed decision-making algorithm of Multi-UAVs cooperative search can be divided into steps, shown in Algorithm 1.

The Simulation Validation and Results Analysis
To verify the effectiveness and feasibility of the proposed method, the simulation platform of Multi-UAVs distributed cooperative search for moving target is established in this section.The dynamic flight trajectory of UAV and motion trajectory of target can be observed in the simulation platform.The conditions of the simulation are designed as follows.
The size of task area is 80 km × 80 km and divided into 80 × 80 grids with the same width and length   = 1 km.Ten stochastic moving targets are distributed in the task area.Three UAVs take off from one base and enter the bottom edge of the search area after a short time.Assume that the speed of UAV is 30∼50 m/s, the speed of moving target is 3∼5 m/s, and the turning angle constraints of UAV and target are −45 ∘ ∼45 ∘ and −180 ∘ ∼180 ∘ , respectively.The simulation results are shown in Figures 6 and 7.
Because the process of UAV searching is dynamic, different steps are selected to describe searching behavior of UAV and the coverage of environment area.In Figure 7, the thin line denotes the search trajectory of UAV and the rectangle denotes the moving trajectory of target.As is shown in Figure 7(a), UAVs can detect some targets at the beginning.With the simulation ongoing, UAVs can continually access individual cells, know about the task area, and confirm whether the target exists.From Figure 7(d), three UAVs have already covered most of the task area and detected more targets than that in Figure 7(a).Figure 8 shows the potential difference variation of pheromone.There are no overlaps of trajectory between the UAVs and no resources to be wasted.The proposed method can avoid collisions and ensure better cooperation between UAVs.All the results above have verified the effectiveness and feasibility of the proposed method.
To further verify the effectiveness of the proposed method in this paper, it is compared with cooperative search method with nonprediction, scanning beam method, and greedy Initialize: Assume that the prediction horizon length is , sample time is   , discrete time is .
Step 1.Let the first item   ( + 1 |  + 1) of control sequence   ( + 1) as the command of output decision.
Step 4. Compute the value of each value function based on formulation (26), ( 27), ( 28), (31), and (32), and obtain total cost value of UAV V  in the prediction horizon length .search method, respectively.There are two aspects of comparison, namely, average numbers of finding targets and average coverage ratio for the task area.Because the initial positions of UAV and moving targets, along with the initialization of area environment, are stochastic in the simulation, twenty simulations are performed, where the length of simulation step is set at 1000.The simulation results are shown in Figures 8 and 9, respectively.According to Figure 8, the average numbers of finding targets for four methods are almost the same at beginning.But, with the increasing simulation steps, it can be found that the proposed method in this paper can effectively find more moving targets in the task area than the other three methods.That has verified the effectiveness of the proposed method on the number of finding targets.From Figure 9, it can be seen that the final average coverage ratio of the proposed method is 44.5%, better than greedy search method and nonprediction cooperation method and a little inferior to scanning beam search method.

Conclusions
As a result of the restriction of the single UAV platform, it might not detect and find moving targets in the complex environment and is even destroyed by enemy units, causing the failure of the entire search and reconnaissance mission.So a distributed cooperative search decision-making method of Multi-UAV based on the moving prediction of targets is proposed in this paper.In this method, detection information of onboard sensors is used to update target existing probability based on Bayesian theory.A Gaussian distribution of target transition probability density function is introduced to calculate prediction probability of moving target existence, and then target probability map can be further updated in real-time.We will build the performance function of cooperative search and adopt receding horizon optimization method to solve and find the maximum control sequence in which searching performance function is to maximize and lead each UAV to effectively search in the task area.

2 InternationalFigure 1 :
Figure 1: The schematic diagram for searching environment and various orientations.

Figure 3 :
Figure 3: The schematic of transition probability density function of maneuvering target.

Figure 4 :
Figure 4: The nonzero competition force and the vector plot between UAVs.

Figure 5 :
Figure 5: The information structure of UAV autonomous cooperation search based on DMPC.